---
title: "Discussion on \"How to Upgrade Your LLM Without Breaking Production: A Replay Testing Guide with ZizkaDB\" | Hashnode"
description: "Discussion on \"How to Upgrade Your LLM Without Breaking Production: A Replay Testing Guide with ZizkaDB\". Sooner or later every team running AI agents in production faces the same decision. The model provider is retiring the version you use. A cheaper model has appeared that might do the job. Latency is t"
url: "https://hashnode.com/posts/how-to-upgrade-your-llm-without-breaking-production-a-replay-testing-guide-with-zizkadb/6afc17497501f8e04b309594"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [How to Upgrade Your LLM Without Breaking Production: A Replay Testing Guide with ZizkaDB](https://mir-zizka.hashnode.dev/how-to-upgrade-your-llm-without-breaking-production-a-replay-testing-guide-with-zizkadb?utm_source=hashnode&utm_medium=feed)

Sooner or later every team running AI agents in production faces the same decision. The model provider is retiring the version you use. A cheaper model has appeared that might do the job. Latency is t

4d ago · 14 min read · mir-zizka.hashnode.dev  
Read the article

## Written by

[Mir Arshad Ali Talpur](https://hashnode.com/@mirarshad)

@mirarshad · Paris, France

  
[View profile](https://hashnode.com/@mirarshad)

Sharing thoughts about future of Ai

Techie, Developer, founder of ZizkaDB

## Responses 2

Markdown supported · ⌘↵ to post

[Dr Srinivas Padmanabhuni](https://hashnode.com/@srinivaspadmanabhuni)

[3d ago](https://hashnode.com/posts/how-to-upgrade-your-llm-without-breaking-production-a-replay-testing-guide-with-zizkadb/6afc17497501f8e04b309594/comment/6afc5971c665382e7b18beec)

The replay method is strongest when it separates model drift from retrieval drift: reusing the same retrieved context makes the comparison fair, but can miss a changed retrieval path. In my work on AiEnsured agentest, an Agentic AI Testing platform, an a17 multi-hop RAG run retrieved the documents containing the answer yet still replied, “Not found in the official documents.” That’s why I’d pair outcome agreement with citation and retrieval checks during disagreement review; your stratified sampling and reviewer-corrected cases offer a practical place to do it.

[Mir Arshad Ali Talpur](https://hashnode.com/@mirarshad)

[3d ago](https://hashnode.com/posts/how-to-upgrade-your-llm-without-breaking-production-a-replay-testing-guide-with-zizkadb/6afc17497501f8e04b309594/comment/6afc5971c665382e7b18beec#reply-6afc69573c0ad99dcb950582)

I agree with it , but the nuance that i am taking here is based on what happened actually not what was supposed to happen or what should have happened, we are audit-ability platform not a framework
