My retriever told me nothing in my four PDFs was about hard negative mining. Page 4 of the embeddings report is about exactly that. On Day 16 I turned 359 chunks into 384-dimension vectors. Today they
Markdown supported · ⌘↵ to post
This is the exact bug I'd expect and most writeups skip it. Good catch showing the raw distances instead of just the fix. One thing worth adding: 1 - distance only looked like similarity because MiniLM happens to emit unit-length vectors. That's a property of that model, not of Chroma or your retriever code. Swap embedding models later and if the new one isn't reliably unit-norm, the scores don't break loudly, they just get quietly worse. Normalizing embeddings yourself right after generation, even when the model already does it, means that failure mode never comes back.
Ahmet Özel
AI Engineer. Computer Vision, RAG and LLM agents.
"A lying score" is the right title, and the specific trap is that cosine distance is not calibrated across queries. A 0.4 on one question and a 0.4 on another do not mean the same thing, so any absolute threshold you pick from one example will misfire on the next.
What made this debuggable for me was checking two things side by side: the rank of the chunk you know is correct, and whether a plain keyword search finds it. If BM25 returns it at rank 2 and vector search does not see it at all, the problem is the embedding or the index rather than the corpus, and you have eliminated half the search space in about a minute.