One GPU, Two LLMsBuilding a production-grade LLM inference platform on EKS with GPU time-slicing, KServe, Knative, and a custom gateway. Finally, debugging everything that broke.6 min read·orhunkupeli.hashnode.devJul 21
The Numbers: Benchmarking My LLM Gateway on a H100Real H100 benchmark of a production LLM gateway vs a naive HuggingFace baseline continuous batching, streaming TTFT, graceful 429s.5 min read·orhunkupeli.hashnode.dev1Jun 21
MVP to Mission-Critical: The Idea Behind My LLM Gateway RewriteHow I rebuilt my LLM gateway from MVP to production: multi-tenant isolation, vLLM inference, budgets, and sub-5ms overhead. Architecture deep dive5 min read·orhunkupeli.hashnode.dev1May 14