---
title: "Comment by Dr Srinivas Padmanabhuni on \"How to Evaluate AI Systems Before Production\" | Hashnode"
description: "The refund example is the strongest part: issuing a refund without approval is a release blocker regardless of the overall score. That is also where traces matt"
url: "https://hashnode.com/posts/how-to-evaluate-ai-systems-before-production/6affecab9b82e9f4dedba4ba/comment/6b017f53fb0672e6a7ab1be7"
---

[Discussion](https://hashnode.com/posts/how-to-evaluate-ai-systems-before-production/6affecab9b82e9f4dedba4ba)

Comment [How to Evaluate AI Systems Before Production](https://hashnode.com/posts/how-to-evaluate-ai-systems-before-production/6affecab9b82e9f4dedba4ba)

[Dr Srinivas Padmanabhuni](https://hashnode.com/@srinivaspadmanabhuni)

[1h ago](https://hashnode.com/posts/how-to-evaluate-ai-systems-before-production/6affecab9b82e9f4dedba4ba/comment/6b017f53fb0672e6a7ab1be7)

The refund example is the strongest part: issuing a refund without approval is a release blocker regardless of the overall score. That is also where traces matter. In one run I reviewed, an agent sent a DELETE through a read-only tool. The server refused it, but nothing in the recorded trace marked the step as denied, so a checker reading the trace had no signal to flag. Policy checks before agent actions help a release review when the refusal is written into the trace as well.

I check recorded agent traces with AiEnsured agentest, 165 tests:

pip install agentest-ai

Developers are welcome to enhance and contribute to it.
