---
title: "Discussion on \"Semantic Caching for LLM Calls in ASP.NET Core: When to Use It and How\" | Hashnode"
description: "Discussion on \"Semantic Caching for LLM Calls in ASP.NET Core: When to Use It and How\". Semantic caching for LLM calls in ASP.NET Core cuts token spend and latency. Learn embeddings, thresholds, invalidation and the traps."
url: "https://hashnode.com/posts/semantic-caching-llm-aspnet-core/6a81a679863aa920fefe8399"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Semantic Caching for LLM Calls in ASP.NET Core: When to Use It and How](https://codingdroplets.com/semantic-caching-llm-aspnet-core?utm_source=hashnode&utm_medium=feed)

Semantic caching for LLM calls in ASP.NET Core cuts token spend and latency. Learn embeddings, thresholds, invalidation and the traps.

Aug 16 · 11 min read · codingdroplets.com  
Read the article

-   [#net](https://hashnode.com/tag/net)
-   [#csharp](https://hashnode.com/tag/csharp)
-   [#aspnet-core](https://hashnode.com/tag/aspnet-core)
-   [#ai](https://hashnode.com/tag/ai)
-   [#llm](https://hashnode.com/tag/llm)
-   [#caching](https://hashnode.com/tag/caching)
-   [#embeddings](https://hashnode.com/tag/embeddings)
-   [#performance](https://hashnode.com/tag/performance)

## Written by

[Coding Droplets](https://hashnode.com/@codingdroplets)

@codingdroplets

  
[View profile](https://hashnode.com/@codingdroplets)

Build, Scale & Master .NET with Clean, Practical Tutorials

## Responses 4

Markdown supported · ⌘↵ to post

[Julian Neagu](https://hashnode.com/@julianneagulab)

[Aug 18](https://hashnode.com/posts/semantic-caching-llm-aspnet-core/6a81a679863aa920fefe8399/comment/6a8453f44e257806c949dd4f)

The cost point hits home. I’ve seen repeated prompts quietly eat through API budgets. Semantic caching helps, but only if you measure the hit rate first.

[Coding Droplets](https://hashnode.com/@codingdroplets)

[Aug 18](https://hashnode.com/posts/semantic-caching-llm-aspnet-core/6a81a679863aa920fefe8399/comment/6a8453f44e257806c949dd4f#reply-6a845dda69281e8b90ec4bc4)

Hi Julian, Thanks! Agreed. Measuring comes first.

One thing I'd add. Hit rate on its own can flatter you. A loose threshold looks like a great result until you check false hits next to it. I measure duplicate rate in shadow mode before building anything, then track both numbers together.

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Aug 17](https://hashnode.com/posts/semantic-caching-llm-aspnet-core/6a81a679863aa920fefe8399/comment/6a831f70e4f4eb6bf6f61ed3)

The threshold tuning is where semantic caching quietly gets dangerous, since a threshold loose enough to catch paraphrases will eventually serve a cached answer to a question that only looks similar but needs a different reply. I keep a small labeled set of near-miss pairs and measure false-hit rate whenever I move the threshold, so a cost win does not turn into a wrong-answer regression. How are you deciding your similarity cutoff, and do you track how often a hit returns a semantically-off answer?

[Coding Droplets](https://hashnode.com/@codingdroplets)

[Aug 17](https://hashnode.com/posts/semantic-caching-llm-aspnet-core/6a81a679863aa920fefe8399/comment/6a831f70e4f4eb6bf6f61ed3#reply-6a832d37e4c3be87f2869cde)

Hi Kartik, Thanks! Threshold tuning is the part most people skip.

No fixed number. The distribution shifts with the embedding model, so I shadow-run the cache for a week, log top scores without serving anything, then label a few hundred real pairs and find where false hits fall off.

On tracking, rephrase rate is the giveaway. If people re-ask more after a hit than a miss, the threshold's too loose.

Thanks again for sharing yours.
