To answer your closing question, escalation is the one I see missed most. Teams score whether the agent got the answer right, sometimes whether it called the right tools, and almost never whether it stopped and handed over when it should have. An agent that confidently finishes a task it should have refused scores well on outcome and is the most dangerous kind.

The must_not_call field is a good way to make that testable. It turns "the agent should not issue a refund here" into a check that fails loudly instead of a hope written in the system prompt.

Treating injection cases as release blockers rather than a score to improve is the right call too. A ninety percent pass rate on prompt injection is not a number anyone should ship.

I also agree with fifty sharp real cases over a thousand generated ones. Generated cases tend to test what the generator imagined, while real ones test what users actually do. How often do you promote cases from the extended set into the frozen core?