FeedDiscussion
Productionising Generative AI with MLflow 3.x: Tracing, Evaluation, and Prompt Optimisation
Generative AI systems fail differently from traditional software. In a normal HTTP service, a production incident often reduces to familiar questions: did the request time out, did the database return
thamu.hashnode.dev12 min read
#lang-en#translation-productionising-generative-ai-mlflow-3x#mlflow#generative-ai#llmops#rag#observability#mlops#ai#artificial-intelligence#python#devops#llm#machine-learning#ai-agents
"The final response can be well-formed and still be wrong" explains why the usual incident playbook does not transfer, and tracing is the right first investment because it turns an unfalsifiable complaint into a specific stage.
The thing I would emphasise alongside it: a trace only helps if the retrieval step records which chunks came back and at what rank, not merely that retrieval happened. Most instrumentations log the span and the latency, and you still cannot tell whether the answer was wrong because the evidence was missing or because it was present and ignored.
Those two need opposite fixes, and it is the cheapest distinction to instrument before you need it rather than after.