---
title: "Comment by Mike Dabydeen on \"Fail-closed authorization for AI agent tool calls, and the audit record behind it\" | Hashnode"
description: "The number I'd watch most in the case study is 60 of 97 benign AgentDojo tasks held for approval, next to 25 of 26 attacks. In the replay, about 3 in 10 holds a"
url: "https://hashnode.com/posts/fail-closed-authorization-for-ai-agent-tool-calls-and-the-audit-record-behind-it/6af6ec76f8d8584368c29d20/comment/6af7000bea6cac6a25c84fe9"
---

[Discussion](https://hashnode.com/posts/fail-closed-authorization-for-ai-agent-tool-calls-and-the-audit-record-behind-it/6af6ec76f8d8584368c29d20)

Comment [Fail-closed authorization for AI agent tool calls, and the audit record behind it](https://hashnode.com/posts/fail-closed-authorization-for-ai-agent-tool-calls-and-the-audit-record-behind-it/6af6ec76f8d8584368c29d20)

[Mike Dabydeen](https://hashnode.com/@firelinks)

[Nov 13](https://hashnode.com/posts/fail-closed-authorization-for-ai-agent-tool-calls-and-the-audit-record-behind-it/6af6ec76f8d8584368c29d20/comment/6af7000bea6cac6a25c84fe9)

The number I'd watch most in the case study is 60 of 97 benign AgentDojo tasks held for approval, next to 25 of 26 attacks. In the replay, about 3 in 10 holds are attacks. In production, where attacks are rare, almost every hold a reviewer sees will be benign, and a queue like that teaches people to click approve. Once that habit sets in, REQUIRE\_APPROVAL protects about as much as ALLOW, whether or not the approved flag gets authenticated.

The audit record from your guide already carries what you need to track this: tool\_name, verdict and the approval fields. If you count held-then-approved per tool and per policy\_version, the tools whose holds are approved nearly every time show you where an argument constraint should replace the human check. Your delete example works that way: a pattern for /workspace/tmp/ could be allowed outright, and only paths outside it would wait for a person.

[Andrey Lesnikov](https://hashnode.com/@justrunme)

[Nov 13](https://hashnode.com/posts/fail-closed-authorization-for-ai-agent-tool-calls-and-the-audit-record-behind-it/6af6ec76f8d8584368c29d20/comment/6af7000bea6cac6a25c84fe9#reply-6af717c7b9e3849d26bdf65d)

Agreed, and the paper reaches a similar conclusion. In the AgentDojo replay every hold comes from the tool-level high-impact list, and 50 of the 60 held benign traces contain exactly one high-impact tool, the task's intended side effect. Often the attack and the legitimate task use the same tool (send\_money, send\_email) and differ only in arguments, so a tool-level approval list can't separate them; fewer holds without losing protection has to come from argument-level rules like your /workspace/tmp example. ASC already supports pattern, deny\_patterns and allow\_hosts per argument, but the frozen policies in the replay were deliberately not re-tuned. The limitations section also notes that human approvers are fallible and fatigable. Counting held-then-approved per tool and policy\_version is a good metric; the current envelope records verdict and policy hash but not the approval outcome, so that would need the approval workflow to log it.

[Mike Dabydeen](https://hashnode.com/@firelinks)

[6d ago](https://hashnode.com/posts/fail-closed-authorization-for-ai-agent-tool-calls-and-the-audit-record-behind-it/6af6ec76f8d8584368c29d20/comment/6af7000bea6cac6a25c84fe9#reply-6afaf529503a002400dbd6e3)

Logging it as a separate record that points back to the decision would leave the envelope as it is: the hash of the held call, the approver, the outcome, and the time from hold to answer. That last field is the early warning. When most approvals for send\_money come back within a couple of seconds, nobody is reading the arguments, and the hold has stopped working well before the approve rate tells you.

For send\_money and send\_email, where the attack and the task share a tool, the argument that usually separates them is the recipient. A payee or domain already in the user's history could pass, and a new recipient is what waits for a person.
