Deterministic context beats generated context — when structure exists

Great question — and honestly, I've stopped trying to draw the line too cleanly up front.

The heuristic I use: if I can write down what "correct" means as a rule — an exact arXiv ID, an author, a date range, a section reference — it's structured, and it belongs in a metadata filter or BM25, not an embedding. The moment "correct" becomes a judgment ("is this paper about the same idea?"), it's genuinely semantic and the model earns its place. The test I apply: can I write the assertion that would fail if the retrieval is wrong? If yes, it's deterministic residue.

But in practice I don't pre-classify every token — that's exactly why hybrid (BM25 + dense, fused with RRF) works. The obvious anchors I filter deterministically; for everything else I run both and let fusion decide. The clean split is the exception, not the rule. I went deeper on the retrieval architecture over an ArXiv corpus here, if useful: blog.deepakgoyal.ai/building-production-rag-arxiv…

And +1 on the silent-failure point — a wrong answer gets caught downstream; wrong context quietly becomes everyone's ground truth.