---
title: "Discussion on \"How a Chunking Bug Taught Me More Than a Model Failure\" | Hashnode"
description: "Discussion on \"How a Chunking Bug Taught Me More Than a Model Failure\". First Principles Log #04 · Building a Climate Risk Agent in Public"
url: "https://hashnode.com/posts/how-a-chunking-bug-taught-me-more-than-a-model-failure/6a623f85edcd423e97e4c863"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [How a Chunking Bug Taught Me More Than a Model Failure](https://aswanisahoo.hashnode.dev/how-a-chunking-bug-taught-me-more-than-a-model-failure?utm_source=hashnode&utm_medium=feed)

First Principles Log #04 · Building a Climate Risk Agent in Public

Jul 23 · 6 min read · aswanisahoo.hashnode.dev  
Read the article

-   [#ai-agents](https://hashnode.com/tag/ai-agents)
-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#climatetech](https://hashnode.com/tag/climatetech)
-   [#build-in-public](https://hashnode.com/tag/build-in-public)
-   [#rag](https://hashnode.com/tag/rag)

## Written by

[Aswani Sahoo](https://hashnode.com/@aswanisahoo)

@aswanisahoo · Rourkela, India

  
[View profile](https://hashnode.com/@aswanisahoo)

Scientific ML and open source: the honest version

B.Tech student at NIT Rourkela working in machine learning and open source. I do scientific ML - weather forecasting and genomics , and ship upstream PRs to projects like PyTorch and Open Climate Fix. This blog is where I write about what I build, what breaks, and what I get wrong.

## Responses 3

Markdown supported · ⌘↵ to post

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Jul 25](https://hashnode.com/posts/how-a-chunking-bug-taught-me-more-than-a-model-failure/6a623f85edcd423e97e4c863/comment/6a6538849008bab1c28e6192)

The k=8 improvement is a useful emergency measure, but the 45-question result shows exactly why you kept it as debt rather than the final fix. Appending the parent caption and column header to each row chunk makes the evidence self-contained instead of relying on the header being retrieved by coincidence. I would add a table-specific eval slice that asserts every numeric cell chunk carries both row and column labels, separating chunk semantic integrity from retrieval quality.

[Aswani Sahoo](https://hashnode.com/@aswanisahoo)

[Jul 28](https://hashnode.com/posts/how-a-chunking-bug-taught-me-more-than-a-model-failure/6a623f85edcd423e97e4c863/comment/6a6538849008bab1c28e6192#reply-6a68ea469a39a23002240edf)

Hey Ahmet, good comment. You're spot on that k=8 was the workaround, not the fix. The actual fix (appending the parent caption to every row chunk) already shipped before this post went live. Code's at chunk.py:171-175 with nine tests covering it.

Quick note on the column header point though. AR6 Table 11.7 doesn't actually have a per-column header row in the extracted text. The 1.5/2/4°C labels only exist in the caption, and cell-to-column mapping is lost once PyMuPDF gives you raw text. So row-atomic split plus caption suffix is as far as you can go without layout-aware parsing.

The idea I'm genuinely taking from this is the metric-separation angle. recall@k mixes up chunk integrity with retrieval quality, so bumping k can hide a chunker bug. I pushed the eval side harder in the follow-up here: [aswanisahoo.hashnode.dev/when-my-evaluation-reach…](https://aswanisahoo.hashnode.dev/when-my-evaluation-reached-zero-errors-i-hired-a-stricter-judge)

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Jul 30](https://hashnode.com/posts/how-a-chunking-bug-taught-me-more-than-a-model-failure/6a623f85edcd423e97e4c863/comment/6a6b76047237856aa3c87072)

That clarification about PyMuPDF losing the 1.5/2/4°C column mapping is important; once geometry is discarded, adding labels later would be reconstruction rather than enrichment. I would make that loss explicit in chunk metadata, for example `table_structure=caption_only`, and route those tables to a layout-aware parser or abstain from cell-level answers. A separate chunk-answerability check can then ask whether the chunk alone preserves every coordinate needed to interpret the value, while recall@k remains purely a retrieval metric.
