---
title: "Comment by Ali Muwwakkil on \"Your AI Agent Makes the Same LLM Call 50 Times a Day — 5 Caching Patterns That Cut Latency and Cost\" | Hashnode"
description: "In our latest accelerator cohort, we tackled this very issue of repetitive LLM calls by diving into caching strategies that can significantly reduce latency and"
url: "https://hashnode.com/posts/your-ai-agent-makes-the-same-llm-call-50-times-a-day-5-caching-patterns-that-cut/69c397276f45bc660cf86a05/comment/69c3a091cf37a8b8e24be2f7"
---

[Discussion](https://hashnode.com/posts/your-ai-agent-makes-the-same-llm-call-50-times-a-day-5-caching-patterns-that-cut/69c397276f45bc660cf86a05)

Comment [Your AI Agent Makes the Same LLM Call 50 Times a Day — 5 Caching Patterns That Cut Latency and Cost](https://hashnode.com/posts/your-ai-agent-makes-the-same-llm-call-50-times-a-day-5-caching-patterns-that-cut/69c397276f45bc660cf86a05)

[Ali Muwwakkil](https://hashnode.com/@enterpriseaileadersh)

[Mar 25](https://hashnode.com/posts/your-ai-agent-makes-the-same-llm-call-50-times-a-day-5-caching-patterns-that-cut/69c397276f45bc660cf86a05/comment/69c3a091cf37a8b8e24be2f7)

In our latest accelerator cohort, we tackled this very issue of repetitive LLM calls by diving into caching strategies that can significantly reduce latency and costs. One key framework we advocate for is the use of a multi-layer cache architecture. This involves: 1. **Memory Caching**: For immediate, short-term storage, tools like Redis or Memcached are extremely effective. They allow for rapid retrieval of frequent queries and can drastically cut down on API requests. 2. **Persistent Caching**: When data needs to survive application restarts, consider using databases like PostgreSQL. We often see teams leverage this for caching results of queries that are frequently repeated over days or weeks. 3. **Query Normalization**: Before hitting the cache, normalize queries to ensure that semantically similar inputs are recognized as identical. This can involve stripping out variable parts or using embeddings to capture the query's essence. 4. **TTL Management**: Implement a smart Time-to-Live (TTL) strategy to balance freshness with efficiency. In enterprise settings, we found that setting TTL based on query patterns and business needs can optimize both relevance and performance. 5. **Custom Middleware**: Building custom middleware for caching logic can help in handling complex scenarios, such as user-specific data or context-aware responses. In practice, these caching layers can be orchestrated to work together, minimizing redundant LLM calls and improving system responsiveness. W

[klement Gunndu](https://hashnode.com/@klementgunnduai)

[Mar 25](https://hashnode.com/posts/your-ai-agent-makes-the-same-llm-call-50-times-a-day-5-caching-patterns-that-cut/69c397276f45bc660cf86a05/comment/69c3a091cf37a8b8e24be2f7#reply-69c3a4986b6384f2a351fad8)

The query normalization layer is the one most teams skip and then wonder why their cache hit rate stays under 30%. Embedding-based similarity matching for near-duplicate prompts is where the real savings compound.
