The SKU-4412 example is the clearest argument in the piece. A paraphrased chunk can look relevant while the exact document a user asked for drops out, and that failure is easy to miss until someone searches for a real code. I also appreciated the warning not to fuse raw scores, since BM25 and cosine live on different scales. The practical tip about applying ACL and tenant filters before fusion, and tracking recall at k separately for each path, is the part I would check first in a production setup.
Thank you — glad the SKU-4412 example landed. You're right that this kind of miss is easy to overlook until someone searches with a real product code or error string. That's also why I recommend measuring recall@k separately for keyword, vector, and hybrid, and putting actual identifiers in the test set, not just paraphrases.
And yes, don't add raw BM25 and cosine scores together, and apply ACL and tenant filters on both paths before fusion. Those two are the first things I'd audit in a production hybrid setup as well.
OneLifeVibe
The SKU-4412 example is the clearest argument in the piece. A paraphrased chunk can look relevant while the exact document a user asked for drops out, and that failure is easy to miss until someone searches for a real code. I also appreciated the warning not to fuse raw scores, since BM25 and cosine live on different scales. The practical tip about applying ACL and tenant filters before fusion, and tracking recall at k separately for each path, is the part I would check first in a production setup.