Ahmet Özel
AI Engineer. Computer Vision, RAG and LLM agents.
The metadata_filter built as a raw interpolated string is the part I'd flag for anyone adapting this beyond the hardcoded example. The moment any piece of that filter comes from user or session data (a tenant id, an account scope) you're building a query string by concatenation with no escaping, same failure class as SQL injection, just in File Search's filter language instead of SQL. We ran into the same "structural check gives false confidence" problem on an adversarial review gate we run internally: confirming a claim has a citation attached is cheap and mechanical, confirming the citation actually supports that specific claim needs a second pass reading both texts side by side, and skipping that pass is exactly how a report that says structurallyValid ships a wrong answer with receipts attached.
"A citation may support a nearby sentence rather than the claim the user cares about" is the failure worth building around, because it survives every check that only asks whether citations exist.
The part I would make explicit in the contract is the unit of attachment. A citation on a whole paragraph is nearly unfalsifiable, while a citation on a single claim can actually be verified with an entailment check.
The stale-source case deserves its own clause too. A document being authoritative and a retrieved passage being current are independent properties, and only the second one degrades on its own while nobody is looking at it.