---
title: "Comment by Rishita Sharma on \"Testing AI-Dependent Code: Mocking LLMs, Evaluating Outputs, and Avoiding Flaky Tests\" | Hashnode"
description: "The invoice total example is exactly how my CI got ignored too. One formatting change from the model and suddenly everyone just reruns until green. Switching to"
url: "https://hashnode.com/posts/testing-ai-dependent-code-mocking-llms-evaluating-outputs-and-avoiding-flaky-tests/6ac33dab2f76360ece243dda/comment/6ac5cd104ca22d4f001e92fc"
---

[Discussion](https://hashnode.com/posts/testing-ai-dependent-code-mocking-llms-evaluating-outputs-and-avoiding-flaky-tests/6ac33dab2f76360ece243dda)

Comment [Testing AI-Dependent Code: Mocking LLMs, Evaluating Outputs, and Avoiding Flaky Tests](https://hashnode.com/posts/testing-ai-dependent-code-mocking-llms-evaluating-outputs-and-avoiding-flaky-tests/6ac33dab2f76360ece243dda)

[Rishita Sharma](https://hashnode.com/@narawaybuilder)

[Oct 7](https://hashnode.com/posts/testing-ai-dependent-code-mocking-llms-evaluating-outputs-and-avoiding-flaky-tests/6ac33dab2f76360ece243dda/comment/6ac5cd104ca22d4f001e92fc)

The invoice total example is exactly how my CI got ignored too. One formatting change from the model and suddenly everyone just reruns until green. Switching to schema checks plus a small normalizer for numbers and dates fixed most of it for me. The part I still struggle with is free text fields. Do you assert anything on those beyond length and required keywords?

[Madhav Bhasin](https://hashnode.com/@madhavbhasin)

[Oct 7](https://hashnode.com/posts/testing-ai-dependent-code-mocking-llms-evaluating-outputs-and-avoiding-flaky-tests/6ac33dab2f76360ece243dda/comment/6ac5cd104ca22d4f001e92fc#reply-6ac6233528eaabc1974c0325)

The normaliser for numbers and dates is exactly the right move — one fix, whole category gone.

For free text: three things I actually assert on — required concept is present (loose, not keyword), length is within a sane range, and no model artefacts (leading "Sure!", markdown fences in plain-text fields). Beyond that I treat it as an evaluation problem, not a unit test problem. Weekly real API runs are where those quality regressions surface, not mocked tests.

[Rishita Sharma](https://hashnode.com/@narawaybuilder)

[Oct 8](https://hashnode.com/posts/testing-ai-dependent-code-mocking-llms-evaluating-outputs-and-avoiding-flaky-tests/6ac33dab2f76360ece243dda/comment/6ac5cd104ca22d4f001e92fc#reply-6ac722b0fb0d064306ade49d)

Treating free text as an evaluation problem instead of a unit test makes sense. The artefact check is a good one, I hadn't thought about asserting on a leading "Sure!". Weekly real runs sounds like the right cadence to me.