---
title: "Discussion on \"How to Evaluate AI Systems Before Production\" | Hashnode"
description: "Discussion on \"How to Evaluate AI Systems Before Production\". Learn how five companies evaluate AI systems before production, from repeatable LLM tests and release gates to live monitoring and agent permissions."
url: "https://hashnode.com/posts/how-to-evaluate-ai-systems-before-production/6affecab9b82e9f4dedba4ba"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [How to Evaluate AI Systems Before Production](https://aiforhealthinframediumcom.hashnode.dev/how-to-evaluate-ai-systems-before-production?utm_source=hashnode&utm_medium=feed)

Learn how five companies evaluate AI systems before production, from repeatable LLM tests and release gates to live monitoring and agent permissions.

1d ago · 4 min read · aiforhealthinframediumcom.hashnode.dev  
Read the article

-   [#ai](https://hashnode.com/tag/ai)
-   [#software-testing](https://hashnode.com/tag/software-testing)
-   [#ai-agents](https://hashnode.com/tag/ai-agents)
-   [#llm](https://hashnode.com/tag/llm)

## Written by

[phildunphy107](https://hashnode.com/@phill007)

@phill007 · Vancouver, Canada

  
[View profile](https://hashnode.com/@phill007)

Turning AI Prototypes Into Real Products

Exploring AI, software architecture, and the journey from prototype to production.

## Responses 1

Markdown supported · ⌘↵ to post

[Dr Srinivas Padmanabhuni](https://hashnode.com/@srinivaspadmanabhuni)

[1h ago](https://hashnode.com/posts/how-to-evaluate-ai-systems-before-production/6affecab9b82e9f4dedba4ba/comment/6b017f53fb0672e6a7ab1be7)

The refund example is the strongest part: issuing a refund without approval is a release blocker regardless of the overall score. That is also where traces matter. In one run I reviewed, an agent sent a DELETE through a read-only tool. The server refused it, but nothing in the recorded trace marked the step as denied, so a checker reading the trace had no signal to flag. Policy checks before agent actions help a release review when the refusal is written into the trace as well.

I check recorded agent traces with AiEnsured agentest, 165 tests:

pip install agentest-ai

Developers are welcome to enhance and contribute to it.
