Defining success per workflow before costing anything is the step that makes this usable, since a RAG grounded-answer bar and a coding-agent passing-tests bar produce completely different denominators. Retry and fallback rate belong in the numerator exactly as you have them, because a silent fallback is where the bill grows without anyone noticing. Are you attributing human-review cost back to the specific model that triggered the review, or averaging it across the workflow?
Defining success per workflow before costing anything is the step that makes this usable, since a RAG grounded-answer bar and a coding-agent passing-tests bar produce completely different denominators. Retry and fallback rate belong in the numerator exactly as you have them, because a silent fallback is where the bill grows without anyone noticing. Are you attributing human-review cost back to the specific model that triggered the review, or averaging it across the workflow?