Answer correctness

Answer correctness asks whether the response agrees with trusted truth or an accepted reference answer. It compares the answer with the reference, not merely with the retrieved context.¹

Tiny example

Use this synthetic airline policy:

Hotel accommodation applies only when:
1. the delay requires an overnight stay, and
2. the airline caused the delay.

The passenger says only that the flight is six hours late. The correct conclusion is: there is not enough information yet. An answer that says “Yes, six hours qualifies” addresses the question, but it is incorrect.

Keep the comparisons separate

Check	Compares the answer with	Question
Faithfulness	Retrieved context	Does the evidence support each claim?
Answer relevance	User question	Did the response address the request?
Answer correctness	Trusted reference	Did it reach an accepted conclusion?

A faithful answer can repeat a stale policy and still be incorrect. A correct answer can come from unsupported model memory and still be unfaithful.

FDE note

Reference answers can also be incomplete. Store the policy version, allow equivalent valid wording, define partial credit, and send disputed or high-impact cases to human review. A model judge is an evaluator, not ground truth.

LangChain, “Evaluate a RAG application”, defines correctness as response versus reference answer and separates it from response relevance and groundedness. ↩