Comment by Ahmet Özel on "Designing the Ingestion Layer of a Production RAG System"

CommentDesigning the Ingestion Layer of a Production RAG System

Ahmet Özel

AI Engineer. Computer Vision, RAG and LLM agents.

The Gemini probe followed by an automatic all-mpnet-base-v2 fallback is resilient at the request level, but risky at the index level. If both providers can write into one collection, a dimension check catches only some mismatches; two models with equal dimensions would still create semantically incompatible vectors. I would pin one embedding fingerprint per ingestion run and collection, then queue failed batches for retry rather than silently changing the vector space mid-index.

AB

Aditya Biranje

AI Researcher

Jul 26

Good point! The fallback is decided once during initialization, so it doesn't switch providers mid-run. That said, pinning an embedding fingerprint to each collection and retrying failed batches is definitely a more production-ready approach. Thanks for the suggestion!

Choosing the provider once at initialization removes the worst mid-run mixing risk. I would still record model ID, revision, dimension, normalization setting, and preprocessing version in the collection metadata, then reject later writers whose fingerprint differs. That makes a rebuild or migration intentional and gives retrieval traces enough information to explain why two index generations behave differently.

Aug 5

Search Hashnode