Locating where retrieval fails is usually more valuable than a single end-to-end accuracy number, because "the answer was wrong" hides whether it was retrieval or generation. I log the retrieved chunks next to the final answer so I can tell which stage actually broke, and it is retrieval far more often than people expect. Good to see a post focused on that diagnosis step specifically.
Kartik N V J K
AI Developer | Making AI reliable, trustworthy & accessible to everyone | Active community contributor
Locating where retrieval fails is usually more valuable than a single end-to-end accuracy number, because "the answer was wrong" hides whether it was retrieval or generation. I log the retrieved chunks next to the final answer so I can tell which stage actually broke, and it is retrieval far more often than people expect. Good to see a post focused on that diagnosis step specifically.