What I think is you should never touch or delete whatever is already streamed to the user as retracting content someone is already reading does more trust damage than the grounding failure itself. Instead, append a targeted correction at the end of the stream, pointing at the specific sentence or claim that didn't check out, using the offsets your grounding evaluator already returns. Not a generic "may contain errors" messages, those get ignored fast. A callout on the exact ungrounded span stays useful. To catch it earlier than a single post-hoc pass, run a cheaper grounding check per streaming window instead of waiting for the full response. You've already embedded the source chunks once at retrieval, so scoring each window against them is just one new embedding plus a similarity comparison, not a fresh retrieval. That gives you an early confidence signal and lets you start preparing the correction UI before the last token lands. For severity, split it into two tiers - a marginal miss gets the inline correction notice and a bad miss (fabricated number, entity not in source_chunks at all) can trigger a background regeneration, and you silently swap in the corrected claim once it's ready, still without blocking the user in the moment.
@askuma
About
TPM building in public. Writing about AI guardrails - architecture, implementation, and what actually works in production. Code-first. No fluff.
Available for
Nothing here yet.