You shipped an AI feature last month. It returns product recommendations, or extracts invoice data, or summarizes tickets — and it mostly works. But every once in a while, a field shows up empty, a nu
Markdown supported · ⌘↵ to post
Spot on. JSON mode just stops syntax errors, but schema enforcement is the only way to actually stop runtime hallucination.
Anthropic is outdated in the article. Anthropic now has JSON outputs and strict tool use that guarantee schema-compliant responses through constrained decoding. The parameter moved from output_format to output_config.format, and the old one is deprecated.
The vendorName/clientName swap you opened with is the sharpest example in here, and it's worth calling out explicitly: strict schema enforcement wouldn't have caught it either. Both fields are valid strings in the right slot, so json_schema with strict:true passes the exact same way json_object mode did. Structural validity and correctness are different guarantees. The only way to catch the swap is a semantic check: a handful of invoices with known field values, asserted after extraction, rerun whenever the prompt or model changes.
Julian Neagu
500+ AI tools shipped solo. Founder of VisionVix.
The cents point is underrated. I’ve had AI outputs behave way better once the schema makes units explicit.
totalAmountCentsleaves a lot less room for weird guesses.