Parsing an LLM's JSON Before the Last Token Arrives

A model that reliably returns valid JSON does not return valid JSON while it is typing. Those are two different guarantees, and most streaming clients quietly assume they are the same one. Constrained