That's a good point, and I agree. CAG definitely introduces its own trade-offs, especially around memory usage and context limits since the relevant knowledge needs to stay loaded in the model's context.
From what I've learned, CAG works best in scenarios where the knowledge base is relatively static or changes infrequently—for example, company policies, product documentation, or internal manuals. In those cases, loading the information once and reusing it can be simpler than maintaining a full RAG pipeline.
For larger or frequently changing datasets, RAG is still the better choice because it retrieves only the relevant chunks instead of keeping everything in memory.
I appreciate you pointing this out—I'll update the article to include the trade-offs of CAG so readers get a more balanced comparison.
Rajesh Killadi
Do you really understand the thing? Ask yourself a question.
I feel like you should've also talked about the new problems CAG brings in. I think it needs way more working memory to hold that cache compared to a RAG setup, since you're basically keeping everything loaded instead of just pulling small chunks when needed.