validateAnswer checks the structured item IDs, but Message remains unconstrained. The model could return an allowed ID while naming a different dish or inventing a price in the prose, and the current validation would still pass.
I would render names and prices from the selected records, or require structured item-level text that can be checked against each referenced candidate. A fixture with valid ItemIDs and a deliberately inconsistent Message would make that boundary visible. It separates “the reference exists” from “the explanation accurately describes the referenced item.”
Ahmet Özel
AI Engineer. Computer Vision, RAG and LLM agents.
validateAnswer checks the structured item IDs, but Message remains unconstrained. The model could return an allowed ID while naming a different dish or inventing a price in the prose, and the current validation would still pass.
I would render names and prices from the selected records, or require structured item-level text that can be checked against each referenced candidate. A fixture with valid ItemIDs and a deliberately inconsistent Message would make that boundary visible. It separates “the reference exists” from “the explanation accurately describes the referenced item.”