---
title: "Discussion on \"Deterministic context beats generated context — when structure exists\" | Hashnode"
description: "Discussion on \"Deterministic context beats generated context — when structure exists\". The prompt is only about 10% of what your LLM actually sees. Most of the other 90% is context you assemble — and when your data has structure, you should build it with code, not another LLM call."
url: "https://hashnode.com/posts/deterministic-context-beats-generated-context/6a747c01e92d9139362c0b68"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Deterministic context beats generated context — when structure exists](https://blog.deepakgoyal.ai/deterministic-context-beats-generated-context?utm_source=hashnode&utm_medium=feed)

The prompt is only about 10% of what your LLM actually sees. Most of the other 90% is context you assemble — and when your data has structure, you should build it with code, not another LLM call.

Aug 6 · 12 min read · blog.deepakgoyal.ai  
Read the article

-   [#artificial-intelligence](https://hashnode.com/tag/artificial-intelligence)
-   [#software-architecture](https://hashnode.com/tag/software-architecture)
-   [#llm](https://hashnode.com/tag/llm)
-   [#ai-agents](https://hashnode.com/tag/ai-agents)
-   [#prompt-engineering](https://hashnode.com/tag/prompt-engineering)

## Written by

[Deepak Goyal](https://hashnode.com/@deepakgoyal-ai)

@deepakgoyal-ai · Faridabad, India

  
[View profile](https://hashnode.com/@deepakgoyal-ai)

Technical Architect · AI-powered systems at scale · Architecture decisions, not opinions

Technical Architect · 20 years in software · Writing about AI architecture, system design, and real decisions at scale

## Responses 2

Markdown supported · ⌘↵ to post

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Aug 7](https://hashnode.com/posts/deterministic-context-beats-generated-context/6a747c01e92d9139362c0b68/comment/6a7631668207567e1a1cb983)

The point about hallucinated context being scarier than a hallucinated answer is the one I keep coming back to. When the model fabricates the inputs and downstream steps treat them as ground truth, the failure is silent and much harder to trace. Your BM25-plus-metadata-filter split for exact identifiers matches what I have seen too. How do you decide the cutoff between the structured residue and the genuinely semantic query in practice?

[Deepak Goyal](https://hashnode.com/@deepakgoyal-ai)

[Aug 10](https://hashnode.com/posts/deterministic-context-beats-generated-context/6a747c01e92d9139362c0b68/comment/6a7631668207567e1a1cb983#reply-6a79a0c88a1938b23dff053d)

Great question — and honestly, I've stopped trying to draw the line too cleanly up front.

The heuristic I use: if I can write down what "correct" means as a rule — an exact arXiv ID, an author, a date range, a section reference — it's structured, and it belongs in a metadata filter or BM25, not an embedding. The moment "correct" becomes a judgment ("is this paper about the same idea?"), it's genuinely semantic and the model earns its place. The test I apply: can I write the assertion that would fail if the retrieval is wrong? If yes, it's deterministic residue.

But in practice I don't pre-classify every token — that's exactly why hybrid (BM25 + dense, fused with RRF) works. The obvious anchors I filter deterministically; for everything else I run both and let fusion decide. The clean split is the exception, not the rule. I went deeper on the retrieval architecture over an ArXiv corpus here, if useful: [blog.deepakgoyal.ai/building-production-rag-arxiv…](https://blog.deepakgoyal.ai/building-production-rag-arxiv-part-1)

And +1 on the silent-failure point — a wrong answer gets caught downstream; wrong context quietly becomes everyone's ground truth.
