Message Assessment UI

The Message Assessment feature is designed to enhance user experience by providing detailed evaluations of chat messages through functionalities like hallucination checks and compliance checks (compliance checks are not yet supported). This feature is beneficial for users in several ways:

• Hallucination Checks: The Hallucination Checks functionality systematically evaluates the level of hallucination in a message by categorising the response into three levels based on its adherence to the retrieved sources. This ensures that users receive accurate and reliable information. Read more about hallucination evaluation here.

• Compliance Checks: Ensures that messages adhere to predefined standards or regulations, which is crucial for maintaining data integrity and meeting organizational compliance requirements (not yet supported).

How to Activate the Hallucination Check

The Hallucination Check feature is automatically enabled if the Hallucination Check option is activated in Internal Knowledge Search.

For activation details, visit: https://unique-ch.atlassian.net/wiki/x/NAD6Gw

How It Works

The message assessment is generated instantly after the assistant's message is completed and appears in the chat interface without requiring a page reload.

Interface

Once the message assessment is available, it appears as an accordion at the bottom of the message.

Assessment Types & Sentiments

Assessments are classified by:

• Type:  String e.g HALLUCINATION

• Sentiment: GREEN, YELLOW, or RED

Positive/Verified assessment	Unverified assessment	Negative assessment

Positive/Verified assessment	Unverified assessment	Negative assessment
Open Assessment with low hallucination level	Open Assessment with medium hallucination level	Open Assessment with high hallucination level

For more information on the meaning of the hallucination level, please visit the Hallucination Evaluation in a RAG-Based Setting | Hallucination Level Metric page.