OOrhun KüpeliJul 21 · 6 min readOne GPU, Two LLMsWhat I was aiming simple in theory: deploy two open-weight LLMs behind a custom gateway, on Kubernetes, infrastructure as code. The contraint that made it interesting was a hard cost cap. One spot GPUorhunkupeli.hashnode.dev
OOrhun KüpeliJun 21 · 5 min readThe Numbers: Benchmarking My LLM Gateway on a H100A couple of weeks ago I wrote about rewriting my LLM gateway to bring it from MVP to production. The architectural claims were; multi-tenancy, hybrid inference , sub-5ms overhead. So I benchmarked it 1orhunkupeli.hashnode.dev
