Aalexeyg377inalexeygolev.hashnode.dev·Jul 8 · 3 min readThe Limits of One Model Per Pod ML PlatformEvery engineering platform is built around trade-offs. For several years the model serving platform followed a simple idea: one model, one Kubernetes pod. It was easy to understand, inexpensive to ope00
Aalexeyg377inalexeygolev.hashnode.dev·Jul 2 · 4 min readThe Most Expensive Word in AI Agents Is “Parallel”I was using a knowledge assistant connected to the systems my team actually works in: Slack, Confluence, Jira, GitHub, internal catalogs, monitoring. It answered questions well. Then I looked at the t00
Aalexeyg377inalexeygolev.hashnode.dev·May 18 · 4 min readTracing in Production: When 1–150ms Turns Into 700ms We run machine learning models in production serving real-time traffic. When our managed serving platform introduced built-in MLflow tracing, we wanted to enable it. Tracing promised visibility into i00
Aalexeyg377inalexeygolev.hashnode.dev·May 13 · 4 min readThe Hidden Cost Curve of “One-Click” ML ObservabilityThe ask was reasonable. We had a high-volume ML serving endpoint and we, engineers and data scientists, wanted visibility into what requests and responses looked like in production. The managed ML ser00