Backend Engineer building distributed systems from scratch
Backend Software Engineer who builds scalable, distributed systems. I write about the real bugs, tradeoffs, and lessons I run into while building things from scratch — mostly backend architecture, concurrency, and systems design.
Markdown supported · ⌘↵ to post
Nice fix. For the one-time re-embed I would avoid rewriting the live index in place: write a new namespace keyed by the new fingerprint, validate row coverage plus a small retrieval regression set, then atomically swap the alias. That gives you a rollback path if normalization or the model revision behaves differently and keeps mixed spaces impossible during migration. Missing model IDs can remain quarantined as legacy data until that migration completes rather than being inferred from dimension.
Ahmet Özel
The 768-versus-384 dimension guard is useful, but model identity needs to be first-class metadata too. Two embedding models can produce vectors with the same dimension and still occupy incompatible spaces, so a dimension check alone can silently pass bad comparisons. I would store model name, revision, and normalization settings with every row, then partition or migrate the index whenever that fingerprint changes.