---
title: "Discussion on \"Where RAG Fails: Understanding the Limitations\" | Hashnode"
description: "Discussion on \"Where RAG Fails: Understanding the Limitations\". RAG makes LLMs smarter, but it isn't magic. Here's where it breaks"
url: "https://hashnode.com/posts/where-rag-fails/6a5d086278ee7d04af52894b"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Where RAG Fails: Understanding the Limitations](https://harsh-prajapati.hashnode.dev/where-rag-fails?utm_source=hashnode&utm_medium=feed)

RAG makes LLMs smarter, but it isn't magic. Here's where it breaks

Jul 19 · 11 min read · harsh-prajapati.hashnode.dev  
Read the article

-   [#ai](https://hashnode.com/tag/ai)
-   [#rag](https://hashnode.com/tag/rag)
-   [#generative](https://hashnode.com/tag/generative)
-   [#generative-ai](https://hashnode.com/tag/generative-ai)
-   [#machine-learning](https://hashnode.com/tag/machine-learning)

## Written by

[Harsh Prajapati](https://hashnode.com/@iamharsh)

@iamharsh

  
[View profile](https://hashnode.com/@iamharsh)

Software Developer at Dentsu | Full Stack Developer | MERN | Next.js | AWS

Software Developer at Dentsu | Full Stack Developer | MERN | Next.js | AWS

## Responses 1

Markdown supported · ⌘↵ to post

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Jul 20](https://hashnode.com/posts/where-rag-fails/6a5d086278ee7d04af52894b/comment/6a5e4bd16b22c6991b944d4e)

The "open-book exam" framing is the cleanest way I have seen to explain why retrieval quality caps everything downstream. The failure mode I hit most often is the one your ambiguous-question point hints at: the retriever confidently returns the top-k chunks even when none of them contain the answer, so the model grounds itself in near-misses. Adding a relevance-threshold check before generation (and letting it return "not in the docs") caught more of my hallucinations than any prompt tweak did.