---
title: "Comment by Darius Tran on \"Why Enterprise AI Needs an Evaluation Strategy Before It Goes Live\" | Hashnode"
description: "The gap between a good demo and real work usually comes down to the test set. Demos use clean inputs, real work has the messy ones. One thing that helped me: bu"
url: "https://hashnode.com/posts/why-enterprise-ai-needs-an-evaluation-strategy-before-it-goes-live/6ad057c6aab428c262426d5c/comment/6ad07c6d51210ccf0ec34103"
---

[Discussion](https://hashnode.com/posts/why-enterprise-ai-needs-an-evaluation-strategy-before-it-goes-live/6ad057c6aab428c262426d5c)

Comment [Why Enterprise AI Needs an Evaluation Strategy Before It Goes Live](https://hashnode.com/posts/why-enterprise-ai-needs-an-evaluation-strategy-before-it-goes-live/6ad057c6aab428c262426d5c)

[Darius Tran](https://hashnode.com/@dariustran)

[Oct 15](https://hashnode.com/posts/why-enterprise-ai-needs-an-evaluation-strategy-before-it-goes-live/6ad057c6aab428c262426d5c/comment/6ad07c6d51210ccf0ec34103)

The gap between a good demo and real work usually comes down to the test set. Demos use clean inputs, real work has the messy ones.

One thing that helped me: build the eval set from cases that actually failed or got escalated, rather than from examples someone invented.

Do you run evals offline before launch, or in shadow mode on live traffic first?

[Sampada Sharma](https://hashnode.com/@techytech)

[Oct 15](https://hashnode.com/posts/why-enterprise-ai-needs-an-evaluation-strategy-before-it-goes-live/6ad057c6aab428c262426d5c/comment/6ad07c6d51210ccf0ec34103#reply-6ad07d75a4eb64b63fb1c227)

That’s a good point. Failed and escalated cases often reveal edge cases that synthetic examples miss. I’d start with offline evals to establish a baseline, then use shadow mode to see how the system handles real-world inputs before letting it take action.