---
title: "Discussion on \"Why How You Split Documents Decides Your RAG Quality\" | Hashnode"
description: "Discussion on \"Why How You Split Documents Decides Your RAG Quality\". Chunking in RAG"
url: "https://hashnode.com/posts/why-how-you-split-documents-decides-your-rag-quality/6ae82a163bf65c4425103568"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Why How You Split Documents Decides Your RAG Quality](https://nagspidey.hashnode.dev/why-how-you-split-documents-decides-your-rag-quality?utm_source=hashnode&utm_medium=feed)

Chunking in RAG

Nov 2 · 9 min read · nagspidey.hashnode.dev  
Read the article

-   [#rag](https://hashnode.com/tag/rag)
-   [#chunking](https://hashnode.com/tag/chunking)
-   [#vector-database](https://hashnode.com/tag/vector-database)
-   [#llm](https://hashnode.com/tag/llm)
-   [#ai-engineering](https://hashnode.com/tag/ai-engineering)

## Written by

[Nagappan S](https://hashnode.com/@imkarthik03)

@imkarthik03 · Chennai, India

  
[View profile](https://hashnode.com/@imkarthik03)

Building scalable products at the intersection of Full Stack Engineering & AI

I build scalable web apps, experiment with AI, and turn complex engineering problems into working products.

## Responses 2

Markdown supported · ⌘↵ to post

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Nov 3](https://hashnode.com/posts/why-how-you-split-documents-decides-your-rag-quality/6ae82a163bf65c4425103568/comment/6ae9e767b115302c2bd3f28b)

Semantic chunks averaging 43 tokens and finishing last at 54% look like the case your hierarchical section is built for. I'm curious how much of the gap to recursive splitting at 512 closes if the semantic splits stay as small retrieval children and the surrounding section gets swapped in before generation.

[Nagappan S](https://hashnode.com/@imkarthik03)

[2d ago](https://hashnode.com/posts/why-how-you-split-documents-decides-your-rag-quality/6ae82a163bf65c4425103568/comment/6ae9e767b115302c2bd3f28b#reply-6af1a555ae5498661777725f)

That's the hierarchical pattern from the post (small children for retrieval, parent section for generation), and it should help: the semantic fragments averaged 43 tokens, which is too thin to answer from, so swapping the section in addresses exactly that. I can't give you a number though. The 69% vs 54% figures come from the FloTorch benchmark, which didn't test that combination. I'd run it on your own corpus and measure end-to-end answer accuracy, not retrieval recall, since Chroma's 91.9% recall for semantic chunking suggests the retrieval side isn't the problem.