Skip to content

Retrieval and RAG·Lesson 55 of 67·expert

Hybrid search: dense plus keyword

Embeddings miss the exact error code, the product name and the rare identifier. BM25 finds them and misses the paraphrase. Run both and fuse the rankings.


In this lesson · 20 min

  • →Name the queries dense retrieval systematically fails
  • →Implement BM25 and explain each of its terms
  • →Fuse two rankings with reciprocal rank fusion
  • →Fuse scores directly, and know why that is harder
  • →Decide when hybrid search is worth its complexity

Dense retrieval is good at meaning and bad at literals. Keyword search is the reverse. Production retrieval runs both, because real queries contain both.

What dense retrieval misses

PythonPython not started

Keyword search handles every one of those, because a rare token is exactly what an inverted index is good at — and fails at the thing dense retrieval does best: finding the passage that answers the question in different words.

BM25

BM25 scores a document by how often the query’s terms appear in it, weighted by how rare each term is across the corpus, and adjusted so long documents do not win by length alone.

PythonPython not started

The three parts are worth naming:

  • IDF — a term appearing in one document out of a million is worth far more than “the”.
  • k1 saturates term frequency: the tenth occurrence of a word adds much less than the second.
  • b normalises by length, so a long document does not score highly merely for being long.

Where each one wins

PythonPython not started

The first query is a bare identifier: BM25 finds it because E4021 is rare, and the dense side has almost nothing to work with. The second is a paraphrase sharing barely a word with its answer — where dense retrieval is supposed to shine, and where a lexical method struggles.

Reciprocal rank fusion

The problem with combining two retrievers is that their scores are not comparable — a cosine similarity of 0.82 and a BM25 score of 7.3 mean nothing to each other. RRF sidesteps that entirely by using only the ranks:

PythonPython not started

c7 wins: third for the dense retriever, first for the keyword one, and the only document both rate highly. That is exactly the behaviour you want — agreement between two different methods is strong evidence.

The constant k controls how much a top rank dominates:

PythonPython not started

Weighted score fusion

If you do want to combine scores, normalise each retriever’s scores into a comparable range first:

PythonPython not started

Try it yourself

Exercise 1

Write rrf(rankings, k=60): score each id by the sum of 1/(k + rank) across every ranking it appears in, and return (id, score) pairs best first.

Your turnPython not started
Exercise 2

Write idf(term, documents) using the BM25 formula, then rarest(query, documents) returning the query term with the highest IDF.

Your turnPython not started

Check yourself

A user pastes 'error E4021' and dense retrieval returns passages about errors in general. Why?

Check yourself

Why is reciprocal rank fusion usually preferred over combining two retrievers' scores directly?

Recap

  • Dense retrieval fails systematically on identifiers, rare names, versions, numbers and negation. Keyword search fails on paraphrase. Real queries contain both.
  • BM25 weights terms by rarity (IDF), saturates repeated terms (k1) and normalises for document length (b).
  • Fuse with reciprocal rank fusion: sum 1/(k + rank) across rankings, k = 60. It needs no comparable scales and no tuning.
  • Weighted score fusion works but needs per-query normalisation, which is query-dependent and has no sensible floor.
  • Hybrid costs one extra index and about ten lines, and removes a whole class of failure.