Good catch. You’re absolutely right, the version in the post only passed the first chunk of a keyword-only note to the reranker. I ran into something similar while building the evals, where a long note was getting scored on the wrong chunk and the model completely missed the answer.

I ended up going with a slightly different fix than what you suggested. Now, a keyword-only note passes all its chunks to the reranker, and the note takes the max score across its chunks. It then adds a few extra reranker calls for longer notes, but it handles cases where the keyword and the actual answer are in different chunks.

Though, I haven't implemented that fixture idea yet. For now, vector search catches everything in my test set, so the keyword-only path isn't really getting exercised. I will add a specific fixture that only keyword search can hit, drop the answer in the last chunk, and see how it holds up.

Appreciate you reading through it so closely!