---
title: "Discussion on \"Semantic Caching in Production: What Breaks When Cosine Similarity Isn't Enough\n\" | Hashnode"
description: "Discussion on \"Semantic Caching in Production: What Breaks When Cosine Similarity Isn't Enough\n\". The 3 AM Page"
url: "https://hashnode.com/posts/semantic-caching-in-production-what-breaks-when-cosine-similarity-isn-t-enough/6a70534568d5fc66a416ca64"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Semantic Caching in Production: What Breaks When Cosine Similarity Isn't Enough](https://hrsvd.hashnode.dev/semantic-caching-in-production-what-breaks-when-cosine-similarity-isn-t-enough?utm_source=hashnode&utm_medium=feed)

The 3 AM Page

Aug 3 · 7 min read · hrsvd.hashnode.dev  
Read the article

-   [#ai](https://hashnode.com/tag/ai)
-   [#llm](https://hashnode.com/tag/llm)
-   [#cosine-similarity](https://hashnode.com/tag/cosine-similarity)
-   [#rag](https://hashnode.com/tag/rag)

## Written by

[Harshvardhan Singh](https://hashnode.com/@hrsvd)

@hrsvd

  
[View profile](https://hashnode.com/@hrsvd)

Writing what matters

I build AI infrastructure, backend systems, and open-source developer tools. This is where I share what I'm building, what I'm learning, and the engineering decisions behind it, from code and system design to the occasional bug that taught me something new.

## Responses 1

Markdown supported · ⌘↵ to post

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Aug 3](https://hashnode.com/posts/semantic-caching-in-production-what-breaks-when-cosine-similarity-isn-t-enough/6a70534568d5fc66a416ca64/comment/6a70eefb376a4ec4ee8a5d46)

The 0.88-0.93 cosine between "cancel my subscription" and "reactivate my subscription" is a perfect example of why raw similarity is a blunt cache key: opposite intent, near-identical vector. Two-stage retrieval with a cross-encoder is what fixed the same false-hit problem for me. The staleness half is harder though: how are you invalidating a cached answer when the underlying source changes but the query text stays identical?