The open-book versus closed-book framing is the clearest way I have seen to explain why retrieval quality caps answer quality regardless of model strength. In practice most RAG wrong answers I see trace back to chunking and retrieval recall, not generation, so I score retrieval hit rate before blaming the LLM. Do you evaluate retrieval and generation as two separate metrics, or judge the final answer end to end?
Kartik N V J K
AI Developer | Making AI reliable, trustworthy & accessible to everyone | Active community contributor
The open-book versus closed-book framing is the clearest way I have seen to explain why retrieval quality caps answer quality regardless of model strength. In practice most RAG wrong answers I see trace back to chunking and retrieval recall, not generation, so I score retrieval hit rate before blaming the LLM. Do you evaluate retrieval and generation as two separate metrics, or judge the final answer end to end?