Hey Ahmet, good comment. You're spot on that k=8 was the workaround, not the fix. The actual fix (appending the parent caption to every row chunk) already shipped before this post went live. Code's at chunk.py:171-175 with nine tests covering it.
Quick note on the column header point though. AR6 Table 11.7 doesn't actually have a per-column header row in the extracted text. The 1.5/2/4°C labels only exist in the caption, and cell-to-column mapping is lost once PyMuPDF gives you raw text. So row-atomic split plus caption suffix is as far as you can go without layout-aware parsing.
The idea I'm genuinely taking from this is the metric-separation angle. recall@k mixes up chunk integrity with retrieval quality, so bumping k can hide a chunker bug. I pushed the eval side harder in the follow-up here: aswanisahoo.hashnode.dev/when-my-evaluation-reach…
That clarification about PyMuPDF losing the 1.5/2/4°C column mapping is important; once geometry is discarded, adding labels later would be reconstruction rather than enrichment. I would make that loss explicit in chunk metadata, for example table_structure=caption_only, and route those tables to a layout-aware parser or abstain from cell-level answers. A separate chunk-answerability check can then ask whether the chunk alone preserves every coordinate needed to interpret the value, while recall@k remains purely a retrieval metric.
Ahmet Özel
AI Engineer. Computer Vision, RAG and LLM agents.
The k=8 improvement is a useful emergency measure, but the 45-question result shows exactly why you kept it as debt rather than the final fix. Appending the parent caption and column header to each row chunk makes the evidence self-contained instead of relying on the header being retrieved by coincidence. I would add a table-specific eval slice that asserts every numeric cell chunk carries both row and column labels, separating chunk semantic integrity from retrieval quality.