---
title: "Discussion on \"RAG as an Attention Optimizer\" | Hashnode"
description: "Discussion on \"RAG as an Attention Optimizer\". Why you can't just dump the whole company wiki into a 2-million-token context window."
url: "https://hashnode.com/posts/rag-as-an-attention-optimizer/6a9ab1a377236e2d34a3705d"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [RAG as an Attention Optimizer](https://devanshumishra.hashnode.dev/rag-as-an-attention-optimizer?utm_source=hashnode&utm_medium=feed)

Why you can't just dump the whole company wiki into a 2-million-token context window.

Sep 4 · 6 min read · devanshumishra.hashnode.dev  
Read the article

-   [#rag](https://hashnode.com/tag/rag)
-   [#llm](https://hashnode.com/tag/llm)
-   [#ai](https://hashnode.com/tag/ai)
-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#system-design](https://hashnode.com/tag/system-design)

## Written by

[Devanshu Mishra](https://hashnode.com/@dmdev261)

@dmdev261

  
[View profile](https://hashnode.com/@dmdev261)

## Responses 1

Markdown supported · ⌘↵ to post

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Sep 4](https://hashnode.com/posts/rag-as-an-attention-optimizer/6a9ab1a377236e2d34a3705d/comment/6a9b36a3228ac1d2356c123a)

Framing retrieval as attention budgeting rather than search explains something that confuses people the first time they measure it: adding more relevant context can make answers worse. Every extra passage competes for the same attention mass, so a correct chunk sitting at rank nine behind eight plausible ones is functionally invisible. It also predicts which distractors hurt most, and it is not unrelated text - near-duplicates do the damage, because an older revision of the same policy looks exactly like the right answer from the model's point of view and there is no signal telling it which copy is authoritative. That is also why lost-in-the-middle is worse in practice than the benchmarks suggest: real corpora are full of near-copies, and the position penalty lands on top of an already ambiguous set.
