Hey Ahmet, good comment. You're spot on that k=8 was the workaround, not the fix. The actual fix (appending the parent caption to every row chunk) already shipped before this post went live. Code's at chunk.py:171-175 with nine tests covering it. Quick note on the column header point though. AR6 Table 11.7 doesn't actually have a per-column header row in the extracted text. The 1.5/2/4°C labels only exist in the caption, and cell-to-column mapping is lost once PyMuPDF gives you raw text. So row-atomic split plus caption suffix is as far as you can go without layout-aware parsing. The idea I'm genuinely taking from this is the metric-separation angle. recall@k mixes up chunk integrity with retrieval quality, so bumping k can hide a chunker bug. I pushed the eval side harder in the follow-up here: https://aswanisahoo.hashnode.dev/when-my-evaluation-reached-zero-errors-i-hired-a-stricter-judge
@aswanisahoo
About
B.Tech student at NIT Rourkela working in machine learning and open source. I do scientific ML - weather forecasting and genomics , and ship upstream PRs to projects like PyTorch and Open Climate Fix. This blog is where I write about what I build, what breaks, and what I get wrong.
Available for
Nothing here yet.