Hybrid Search for RAG: Combining BM25 and Vector Search for Better Retrieval
If you have ever watched a RAG pipeline confidently return the wrong chunk, there is a good chance the problem was not your prompt or your model. It was your retriever. Vector search alone has a speci
Responses
Markdown supported · ⌘↵ to post
The framing of RRF sidestepping score normalisation by throwing away the raw scores and keeping only rank position is the cleanest explanation of why it holds up when BM25 floats and cosine values live in incompatible spaces. In practice I have seen the k constant matter more than people expect: at 60 it flattens the top so heavily that a chunk BM25 alone would have nailed on an exact identifier can get diluted by the dense list, so I sometimes drop k when exact-match queries dominate. Have you tried measuring recall separately on the exact-identifier queries versus the conceptual ones, since RRF averaged over both can hide a regression on the code and error-code lookups that motivated adding BM25 in the first place?