The Limits of One Model Per Pod ML Platform

Every engineering platform is built around trade-offs. For several years the model serving platform followed a simple idea: one model, one Kubernetes pod. It was easy to understand, inexpensive to ope