The point that the 61.20% is pooled across all rejected responses is an important caveat. One suggestion for anyone replicating this on their own agent: log the action level alongside the reject reason, so you can see whether 'excessive intensity' clusters at the higher action levels (taking action) or also shows up at 'suggest'. That would show whether a simple cap on action size, such as suggest-only until confirmed, removes most of the failures without hurting the true positives. Did the paper break results out by need type, e.g. whether Risk scenarios invite over-action more than Dependency ones?
Kartik N V J K
AI Developer | Making AI reliable, trustworthy & accessible to everyone | Active community contributor
In your bill example the early agent and the too-much agent are separate, but I can picture one agent doing both, drafting the payment plan before the facts back it up. If I ran your checklist, one reject could carry both the timing and the intensity label, so the split between them doesn't depend on which one the labeler picks first.