Tag feed

#llm-gateway

5 posts0 followers

Explore Hashnode

Alternatives

OKOrhun Küpeliorhunkupeli.hashnode.devJun 21 · 5 min read

The Numbers: Benchmarking My LLM Gateway on a H100

A couple of weeks ago I wrote about rewriting my LLM gateway to bring it from MVP to production. The architectural claims were; multi-tenancy, hybrid inference , sub-5ms overhead. So I benchmarked it

0

ARAshutosh Rathashublog.hashnode.devMay 26 · 7 min read

Launching LLM Gateway: Building a Control Plane for LLM Reliability

Repo: ashutosh-rath02/llm-gateway Why Build Another Layer Around LLMs? Most teams do not begin with a gateway. They begin with a single feature, a provider SDK, one prompt, and a direct API call insi

0

OKOrhun Küpeliorhunkupeli.hashnode.devMay 14 · 5 min read

MVP to Mission-Critical: The Idea Behind My LLM Gateway Rewrite

Several months ago I decided to play around with my first LLM gateway prototype which I simply used LiteLLM with some benefits on top. Then I did the math to find out how far it was from the productio

0

JKJangwook Kimeffloow.hashnode.devApr 16 · 10 min read

LiteLLM: One Proxy for 140+ LLMs — Setup & Cost Guide

Why Every Production AI Team Needs an LLM Gateway Enterprise spending on LLM APIs surpassed $8.4 billion in 2026. The teams burning the most aren't the ones with the largest models — they're the ones without a routing layer between their code and the...

0

SDSylvester Dasblog.minifyn.comMar 30 · 4 min read

Build a Real-Time Medical Transcription Analysis App with AssemblyAI and LLM Gateway

Medical transcription is a critical process, but manual methods are slow and prone to error. Real-time analysis of medical conversations can significantly improve patient care, streamline documentatio

0

#llm-gateway

Search Hashnode

#llm-gateway

Explore Hashnode

The Numbers: Benchmarking My LLM Gateway on a H100

Launching LLM Gateway: Building a Control Plane for LLM Reliability

MVP to Mission-Critical: The Idea Behind My LLM Gateway Rewrite

LiteLLM: One Proxy for 140+ LLMs — Setup & Cost Guide

Build a Real-Time Medical Transcription Analysis App with AssemblyAI and LLM Gateway

Trending tags this week