Kevin Kirui

Testing what the reviewer actually sees, separately from the model output, is the part I'm taking from this. Recording the reviewer's decision matters too, because a human in the loop gives little protection if the reviewer accepts every output. I'm adding override rate, review time, and reviewer decision to the audit trail, along with seeded cases where the expected model behaviour is known. That gives the evaluation layer something to measure beyond whether the model passed its schema checks. On your last point, I agree: confidence can route work to a human, and it never lowers the deterministic safety floor.

Agreed on the inline versus async split. I'm limiting inline tests to the ones that can block or constrain downstream action: schema validity, required fields, safety-floor violations, and known injection patterns. The heavier behavioural tests run asynchronously against logged cases, since many of them need reviewer outcomes or longitudinal workflow data. Your point about confidence versus workflow authority holds too. Confidence can decide whether a case needs human review. It never gives the model authority to bypass a deterministic constraint. I'm treating that separation as a core design rule for the system.