---
title: "Comment by netproxy on \"Testing AI-Dependent Code: Mocking LLMs, Evaluating Outputs, and Avoiding Flaky Tests\" | Hashnode"
description: "Good breakdown of the tradeoffs. One middle ground worth mentioning: for the live-API integration tests, passing a fixed seed (where the provider supports it) t"
url: "https://hashnode.com/posts/testing-ai-dependent-code-mocking-llms-evaluating-outputs-and-avoiding-flaky-tests/6ac33dab2f76360ece243dda/comment/6ac42d3e350baa25b6591b81"
---

[Discussion](https://hashnode.com/posts/testing-ai-dependent-code-mocking-llms-evaluating-outputs-and-avoiding-flaky-tests/6ac33dab2f76360ece243dda)

Comment [Testing AI-Dependent Code: Mocking LLMs, Evaluating Outputs, and Avoiding Flaky Tests](https://hashnode.com/posts/testing-ai-dependent-code-mocking-llms-evaluating-outputs-and-avoiding-flaky-tests/6ac33dab2f76360ece243dda)

[netproxy](https://hashnode.com/@argumentmoney8117)

[Oct 5](https://hashnode.com/posts/testing-ai-dependent-code-mocking-llms-evaluating-outputs-and-avoiding-flaky-tests/6ac33dab2f76360ece243dda/comment/6ac42d3e350baa25b6591b81)

Good breakdown of the tradeoffs. One middle ground worth mentioning: for the live-API integration tests, passing a fixed seed (where the provider supports it) together with temperature 0 gets you much closer to deterministic without going fully mocked. It's not a guarantee - the same seed can still drift across model versions - but it noticeably reduces the retry count. Pairs well with the normalization point in the comments.

[Madhav Bhasin](https://hashnode.com/@madhavbhasin)

[Oct 7](https://hashnode.com/posts/testing-ai-dependent-code-mocking-llms-evaluating-outputs-and-avoiding-flaky-tests/6ac33dab2f76360ece243dda/comment/6ac42d3e350baa25b6591b81#reply-6ac6236edf1531d125998a5c)

Good addition — temperature=0 plus a fixed seed is a useful middle ground, especially for integration tests where you want some real-API coverage but less noise. The caveat you flagged is the right one: seed behaviour isn't guaranteed across model versions, so it's a noise reducer not a flakiness eliminator. Still worth doing. I'd pair it with snapshotting the model version in the test so a version bump is a visible signal rather than a silent source of drift.