Natural Language Inference (NLI)

Natural Language Inference (NLI)—also known historically as Recognizing Textual Entailment (RTE)—is a foundational benchmark in Natural Language Processing that tests whether a model can determine the logical relationship between two statements.

Given a pair of sentences consisting of a Premise ($P$) and a Hypothesis ($H$), an NLI system must classify their relationship into one of three mutually exclusive categories:

1. Entailment: If the premise is true, the hypothesis must be true.
2. Contradiction: If the premise is true, the hypothesis must be false.
3. Neutral: The hypothesis might be true or false; the premise provides insufficient evidence to confirm or deny it.

Canonical Examples

Premise: "Two golden retrievers are chasing a frisbee across a grassy park."

┌────────────────────────────────────────────────────────┬───────────────┐
│ Hypothesis                                             │ Relationship  │
├────────────────────────────────────────────────────────┼───────────────┤
│ "There are dogs playing outside."                      │ Entailment    │
│ "The park is completely empty of animals."             │ Contradiction │
│ "The dogs belong to a professional frisbee trainer."   │ Neutral       │
└────────────────────────────────────────────────────────┴───────────────┘

Notice that Neutral does not mean the hypothesis is impossible—it simply means the truth cannot be established strictly from the given premise alone.

Benchmark Datasets: SNLI and MNLI

The emergence of modern deep NLI models was propelled by two large-scale annotated corpora:

1. SNLI (Stanford Natural Language Inference)

• Scale: $570{,}000$ human-written sentence pairs.
• Domain: Derived strictly from image captioning datasets (Flickr30k). While large, the linguistic style is homogeneous and primarily grounded in physical scenes.

2. MNLI (Multi-Genre Natural Language Inference)

• Scale: $433{,}000$ sentence pairs.
• Domain: Spans 10 distinct genres of spoken and written text (fiction, government reports, telephone conversations, letters, Slate magazine articles).
• Matched vs. Mismatched Test Sets: Evaluates both within-domain generalization (matched) and zero-shot cross-genre robustness on unseen genres (mismatched).

Cross-Encoder Architecture for NLI

Because NLI requires fine-grained token-to-token semantic alignment (e.g., verifying quantifiers, temporal markers, and negations), state-of-the-art models employ Cross-Encoder transformers (such as RoBERTa-large or DeBERTa-v3):

Input: [CLS] Premise Tokens [SEP] Hypothesis Tokens [SEP]
                               │
                [ Deep DeBERTa Transformer Layers ]
                               │
                       [ [CLS] Hidden State ]
                               │
                     [ Linear Layer + Softmax ]
                               │
               ┌───────────────┼───────────────┐
               ▼               ▼               ▼
          Entailment     Contradiction      Neutral

The probability distribution over the three classes is:

$$P(\text{class} \mid P, H) = \text{Softmax}\left(\mathbf{W} \cdot \mathbf{h}_{\text{[CLS]}} + \mathbf{b}\right)$$

Trained using standard multi-class cross-entropy loss against human consensus annotations.

Zero-Shot Text Classification Using NLI

One of the most powerful real-world applications of NLI is Zero-Shot Topic Classification, introduced by Yin et al. (2019). Any arbitrary classification problem can be reformulated as an entailment query without requiring labeled training examples for the target classes!

The Transformation Method

Given an unlabeled input text $x$ and a candidate set of labels ${c_1, c_2, \dots, c_k}$:

1. Set the input text as the Premise: $P = x$.
2. Formulate a Hypothesis template for each candidate label: $$H_k = \text{“This text is about } c_k\text{.”}$$
3. Pass each $(P, H_k)$ pair through a pretrained NLI model.
4. Extract the probability of the Entailment logit: $$\text{Score}(c_k) = P(\text{Entailment} \mid P, H_k)$$
5. Normalize the entailment scores across all candidate labels via softmax.

Input: "The Federal Reserve raised interest rates by 25 basis points today."

Hypothesis 1: "This text is about sports."     ──► Entailment Score: 0.01
Hypothesis 2: "This text is about finance."    ──► Entailment Score: 0.98  <-- Selected!
Hypothesis 3: "This text is about healthcare." ──► Entailment Score: 0.02

Practical Python Implementation with Hugging Face

from transformers import pipeline

# Load a zero-shot classification pipeline powered by a DeBERTa NLI model
classifier = pipeline(
    "zero-shot-classification",
    model="MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli"
)

sequence = "SpaceX successfully launched the Starship spacecraft into orbit, demonstrating booster catch recovery."
candidate_labels = ["aerospace technology", "culinary arts", "macroeconomics", "entertainment"]

result = classifier(
    sequence,
    candidate_labels,
    hypothesis_template="This article is about {}."
)

for label, score in zip(result["labels"], result["scores"]):
    print(f"{label:<25} Confidence: {score * 100:.2f}%")

NLI for Hallucination Detection in RAG

In modern Retrieval-Augmented Generation (RAG), NLI models serve as automated factuality evaluators:

• Premise: The retrieved context passages.
• Hypothesis: The generated statement emitted by the LLM.
• If the NLI model predicts Contradiction or Neutral, the LLM has generated unsupported claims or hallucinated details not grounded in the source documentation.

Key Takeaways

• NLI classifies directional semantic entailment across Premise-Hypothesis pairs into Entailment, Contradiction, or Neutral.
• Cross-encoders (DeBERTa) achieve human-level accuracy on MNLI benchmarks by computing all-to-all attention across premise and hypothesis.
• Reformulating classification as an NLI problem unlocks zero-shot topic classification and automated hallucination auditing in RAG systems.