---
title: "Discussion on \"I Let My LLM Guess a Number My Own Code Already Knew\" | Hashnode"
description: "Discussion on \"I Let My LLM Guess a Number My Own Code Already Knew\". A cosine similarity score and an LLM's own guess landed on the same job card — and disagreed. Here's the bug, the fix, and why one number should win."
url: "https://hashnode.com/posts/i-let-my-llm-guess-a-number-my-own-code-already-knew/6a64d556ad9b47ca8f0d45d6"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [I Let My LLM Guess a Number My Own Code Already Knew](https://thestacktrace.hashnode.dev/i-let-my-llm-guess-a-number-my-own-code-already-knew?utm_source=hashnode&utm_medium=feed)

A cosine similarity score and an LLM's own guess landed on the same job card — and disagreed. Here's the bug, the fix, and why one number should win.

Jul 25 · 6 min read · thestacktrace.hashnode.dev  
Read the article

-   [#rag](https://hashnode.com/tag/rag)
-   [#llm](https://hashnode.com/tag/llm)
-   [#kotlin](https://hashnode.com/tag/kotlin)
-   [#ai](https://hashnode.com/tag/ai)
-   [#springboot](https://hashnode.com/tag/springboot)

## Written by

[Ashish Babu](https://hashnode.com/@ashish-babu)

@ashish-babu · Chennai, India

  
[View profile](https://hashnode.com/@ashish-babu)

Backend Engineer building distributed systems from scratch

Backend Software Engineer who builds scalable, distributed systems. I write about the real bugs, tradeoffs, and lessons I run into while building things from scratch — mostly backend architecture, concurrency, and systems design.

## Responses 4

Markdown supported · ⌘↵ to post

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Jul 25](https://hashnode.com/posts/i-let-my-llm-guess-a-number-my-own-code-already-knew/6a64d556ad9b47ca8f0d45d6/comment/6a6538250e35902e78232c4d)

The 768-versus-384 dimension guard is useful, but model identity needs to be first-class metadata too. Two embedding models can produce vectors with the same dimension and still occupy incompatible spaces, so a dimension check alone can silently pass bad comparisons. I would store model name, revision, and normalization settings with every row, then partition or migrate the index whenever that fingerprint changes.

[Ashish Babu](https://hashnode.com/@ashish-babu)

[Jul 26](https://hashnode.com/posts/i-let-my-llm-guess-a-number-my-own-code-already-knew/6a64d556ad9b47ca8f0d45d6/comment/6a6538250e35902e78232c4d#reply-6a659fdabf16dd0ac8d41665)

Good catch, and honestly you're right that I only check dimension right now, not model identity. Didn't think about the case where two different models spit out the same dim but totally different vector spaces — that's a real hole, not just a theoretical one since I'm already juggling Ollama locally vs HuggingFace in prod.

Storing model name + revision alongside each row makes sense. Only thing I'd add on my end is I'd probably also need a one-time re-embed job for anything already in the table before I flip a hard check on, otherwise I lock myself out of my own data.

Appreciate you actually reading closely enough to catch this, going to go fix it.

[Ashish Babu](https://hashnode.com/@ashish-babu)

[Jul 26](https://hashnode.com/posts/i-let-my-llm-guess-a-number-my-own-code-already-knew/6a64d556ad9b47ca8f0d45d6/comment/6a6538250e35902e78232c4d#reply-6a65aaf24541463347b15762)

Fixed this, added model identity tracking, guards against both mismatched and missing model IDs now, existing embeddings force a re-embed instead of guessing.

Here's the commit: [github.com/ashish-babu-03/ProximaHire/commit/31b1…](https://github.com/ashish-babu-03/ProximaHire/commit/31b1c4a20d4775c68d6063c6997aaeec5e90b0a9)

Appreciate you catching it :)

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Jul 30](https://hashnode.com/posts/i-let-my-llm-guess-a-number-my-own-code-already-knew/6a64d556ad9b47ca8f0d45d6/comment/6a6b7630693f946eea93793a)

Nice fix. For the one-time re-embed I would avoid rewriting the live index in place: write a new namespace keyed by the new fingerprint, validate row coverage plus a small retrieval regression set, then atomically swap the alias. That gives you a rollback path if normalization or the model revision behaves differently and keeps mixed spaces impossible during migration. Missing model IDs can remain quarantined as legacy data until that migration completes rather than being inferred from dimension.
