RAG Hallucination Detection with Groundedness Scoring

Retrieval-Augmented Generation (RAG) systems are often sold as the cure for Large Language Model (LLM) hallucinations. The promise is simple: give the model access to a private knowledge base, and it will only speak what it finds there. In practice, RAG systems frequently generate plausible-sounding but factually incorrect answers, or they ignore the retrieved context entirely in favor of their pre-training data. This is not a bug in the retrieval step; it is a failure of faithfulness in the generation step.

To detect and mitigate these hallucinations, we must move beyond simple keyword matching and implement Groundedness Scoring. This technique uses Natural Language Inference (NLI) to mathematically verify whether the generated answer is logically entailed by the retrieved context.

The Mechanism of Hallucination in RAG

To understand how to detect hallucinations, we must first understand their mechanism. In a RAG pipeline, the LLM receives two primary inputs: the user’s query and a set of retrieved document chunks. The model then generates a response.

Hallucinations occur when the generated tokens are not supported by the context. This happens because LLMs are probabilistic next-token predictors optimized for fluency, not truth. When the context is ambiguous, noisy, or missing critical information, the model’s internal weights (pre-trained on vast internet corpora) dominate the output, leading to "confabulation."

In evaluation frameworks, we distinguish between two types of errors:

1. Answer Relevance: Does the answer address the user’s question?
2. Faithfulness (Groundedness): Is the answer strictly derived from the provided context?

Standard metrics like BLEU or ROUGE measure surface-level similarity, which is insufficient for faithfulness. A model can paraphrase the context perfectly but still introduce subtle factual errors. Therefore, we need a mechanism that checks logical entailment. In modern AI systems, NLI has become the standard approach for verifying entailment between a premise and a hypothesis, providing a rigorous check on faithfulness that semantic similarity alone cannot offer.

Natural Language Inference as a Verifier

Natural Language Inference (NLI), also known as Recognizing Textual Entailment (RTE), is a classic NLP task. Given two sentences, the model determines the relationship between them:

• Entailment: The second sentence (hypothesis) is true if the first sentence (premise) is true.
• Contradiction: The second sentence is false if the first sentence is true.
• Neutral: The truth of the second sentence cannot be determined from the first.

In the context of RAG, we treat the retrieved context as the premise and the generated answer (or specific claims within it) as the hypothesis. If an NLI model predicts "Entailment," the answer is grounded. If it predicts "Neutral" or "Contradiction," the answer contains hallucinations or unsupported extrapolations.

This approach shifts the problem from "does the text look similar?" to "is the logic valid?" It is crucial to distinguish NLI from semantic similarity. While cosine distance measures vector proximity, it cannot distinguish between a correct entailment and a contradictory statement that uses similar vocabulary. For instance, "The cat is on the mat" and "The cat is not on the mat" may have high semantic similarity but opposite logical truth values. NLI models, such as those based on the DeBERTa or RoBERTa architectures, are trained to handle negation and complex reasoning patterns, making them reliable verifiers for structured evaluation. They achieve state-of-the-art performance on benchmarks like SNLI and MultiNLI, allowing us to detect subtle logical gaps that simple embedding models miss.

Implementing Groundedness Scoring

A single answer may contain multiple claims. Evaluating the entire answer as one block can lead to false positives if one minor detail is unsupported while the core answer is correct. Therefore, the effective approach is to decompose the answer into atomic claims and evaluate each one.

Here is a practical implementation using Python and the transformers library, which provides pre-trained NLI models. Note that for true NLI scoring, we use the model's classification head rather than just cosine similarity.

from transformers import pipeline
import torch
 
# Load a pre-trained NLI pipeline (DeBERTa-large trained on MultiNLI)
# This model detects logical relationships in text via classification
nli_pipeline = pipeline("text-classification", model="cross-encoder/nli-deberta-v3-base")
 
def compute_groundedness_score(premise: str, hypothesis: str) -> float:
    """
    Computes a groundedness score between 0 and 1 using NLI classification.
    1.0 means the hypothesis is fully entailed by the premise.
    0.0 means the hypothesis is contradictory or completely unsupported.
    """
    
    # Input format for cross-encoder NLI models: [premise, hypothesis]
    inputs = [premise, hypothesis]
    
    # Get prediction probabilities
    # The output typically contains scores for 'entailment', 'neutral', 'contradiction'
    result = nli_pipeline(inputs)
    
    # Extract the probability of 'entailment'
    # Note: Key names depend on the specific model's label mapping
    entailment_score = result[0]['entailment']
    
    return entailment_score
 
# Example Usage
query = "Who developed the RAG framework?"
context = "Retrieval-Augmented Generation (RAG) was introduced by Lewis et al. in 2020. It combines pre-trained retrieval with generative models."
answer = "The RAG framework was developed by Lewis et al. in 2020."
 
# Combine query and context for better grounding context
premise = f"{query} {context}"
hypothesis = answer
 
score = compute_groundedness_score(premise, hypothesis)
print(f"Groundedness Score: {score:.4f}")

Refining with Classification Heads

The previous example uses a cross-encoder pipeline, which is efficient for pairwise scoring. For higher accuracy, you should ensure you are using a classification variant (e.g., BertForSequenceClassification) that exposes logits directly if you are building a custom training loop.

A robust scoring function might look like this when accessing logits directly:

def strict_nli_score(premise: str, hypothesis: str, model, tokenizer) -> float:
    inputs = tokenizer(premise, hypothesis, return_tensors="pt", truncation=True, max_length=512)
    with torch.no_grad():
        outputs = model(**inputs)
        logits = outputs.logits
    
    # Extract probability of Entailment (class 0 in many NLI datasets)
    # Note: Class indices depend on the specific model architecture
    entailment_prob = torch.softmax(logits, dim=-1)[0][0] 
    
    return entailment_prob.item()

This returns a value between 0 and 1, representing the confidence that the answer is grounded in the context.

Thresholding and Actionable Outcomes

A score is only useful if you act on it. In a production RAG system, you should define a threshold for groundedness. As suggested by industry best practices for production RAG systems, such thresholds help balance recall and precision:

• Score > 0.9: High confidence. Return the answer.
• 0.5 < Score ≤ 0.9: Ambiguous. Return the answer but add a disclaimer: "This information is based on retrieved documents, but may not be fully complete."
• Score ≤ 0.5: Low confidence. Do not return the answer. Trigger a fallback mechanism, such as returning a "I don't know" message or re-running the retrieval with different parameters.

Trade-offs and Limitations

It is important to note that NLI-based verification is not perfect. NLI models can struggle with complex reasoning, negation, or implicit information. If the context says "Apple is not a fruit" (in a specific programming context) and the answer says "Apple is a fruit," the NLI model might correctly flag this as a contradiction, even if the user meant the fruit. Conversely, if the context is sparse, the model may predict "Neutral" for a correct but unmentioned detail, lowering the score unnecessarily.

Therefore, groundedness scoring should be used as a signal, not an absolute truth detector. Combining it with other metrics, such as context recall (measuring how much of the relevant information was retrieved), provides a more holistic view of RAG health.

Conclusion

Hallucinations in RAG systems are a result of the model prioritizing fluency over fidelity to the retrieved context. By implementing Groundedness Scoring via Natural Language Inference, engineers can quantify the faithfulness of generated answers. This transforms hallucination detection from a subjective review process into an automated, scalable metric. While not infallible, NLI-based verification provides the necessary mechanism to filter out hallucinations and improve the trustworthiness of AI-driven applications.