---
title: "Discussion on \"Build a Solid RAG System in One Weekend (2026 Edition)\" | Hashnode"
description: "Discussion on \"Build a Solid RAG System in One Weekend (2026 Edition)\". The four decisions that actually determine whether your RAG system works - chunking, retrieval, reranking, and evaluation with the trade-offs the tutorials usually skip."
url: "https://hashnode.com/posts/build-a-solid-rag-system-in-one-weekend-2026-edition/6aa04392e15956ac260ee923"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Build a Solid RAG System in One Weekend (2026 Edition)](https://truenorthbits.hashnode.dev/build-a-solid-rag-system-in-one-weekend-2026-edition?utm_source=hashnode&utm_medium=feed)

The four decisions that actually determine whether your RAG system works - chunking, retrieval, reranking, and evaluation with the trade-offs the tutorials usually skip.

Sep 8 · 9 min read · truenorthbits.hashnode.dev  
Read the article

-   [#rag](https://hashnode.com/tag/rag)
-   [#python](https://hashnode.com/tag/python)
-   [#vector-database](https://hashnode.com/tag/vector-database)
-   [#bm25](https://hashnode.com/tag/bm25)

## Written by

[Viv Chaudhary](https://hashnode.com/@chuadharyviv)

@chuadharyviv · Bangalore, India

  
[View profile](https://hashnode.com/@chuadharyviv)

Platform & Infrastructure Engineer working in banking, focused on Kubernetes, and AI

Platform & Infrastructure Engineer working in banking sector, focused on Kubernetes, enterprise storage (NetApp, CEPH), and Cloud (AWS, GCP). I build agentic AI systems for real infrastructure workflows where the constraints are compliance, audit trails, and legacy systems, not clean greenfield demos. I write about developer experience and tooling from that vantage point: what agentic AI, MCP tooling, and automation actually look like inside a regulated environment, not just in a polished tutorial.

## Responses 2

Markdown supported · ⌘↵ to post

[Viv Chaudhary](https://hashnode.com/@chuadharyviv)

[Sep 12](https://hashnode.com/posts/build-a-solid-rag-system-in-one-weekend-2026-edition/6aa04392e15956ac260ee923/comment/6aa595ff733748c34e049436)

On provenance: the chunk metadata (page\_id, title, version, url, breadcrumb, section) is captured correctly at embed time, but the chunk id itself is page\_id::chunk\_index with no version in the key. So an incremental reindex overwrites a chunk in place, and any query\_audit\_log row that cited that chunk id before the update now resolves to content that didn't exist when the answer was generated. The audit trail is provenanced going forward, not backward. Fix is to make the id page\_id::version::chunk\_index (or soft-supersede old chunks instead of overwriting) and store enough in the audit row to reconstruct what was actually cited without a live join against current index state. Doing this after the fact means a full reindex since it changes the primary key shape better to bake it in now.

On ordering: fair, and I can't actually defend it either way, the repo has a single squashed commit, so there's no way to show whether the golden set predates the chunking parameters (max\_tokens=400, overlap\_tokens=60) or came after. That ambiguity is exactly the "can't tell a reranker win from noise" problem. Going forward I'll either keep real commit history or state the eval-before-chunker ordering explicitly in the design doc, since the git log can't attest to it.

Will get both fixed, versioned chunk ids first since it's the one that gets more expensive the longer it waits.

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Sep 11](https://hashnode.com/posts/build-a-solid-rag-system-in-one-weekend-2026-edition/6aa04392e15956ac260ee923/comment/6aa39bf3146a5356fc7ecd0e)

Putting evaluation in the same tier as chunking and retrieval is what most weekend guides get wrong, and the regulated-environment vantage point makes the argument land harder. Order matters too: if the eval set does not exist before the chunker is tuned, every later decision is made on vibes and you cannot distinguish a reranker improvement from noise.

One addition from the compliance angle you write from. Audit trails want per-stage provenance, which is a constraint on chunk design rather than a logging afterthought. If a chunk cannot be traced back to a document version, a section and a page, the answer is unauditable no matter how good the retrieval was, and retrofitting that later means reindexing everything.