Gateway-level observability is underrated, since the gateway is the one place you see every model call, retry, and fallback in one stream. What made mine useful was logging the decision, which model, why, and what it cost, not just latency and status. Are you capturing routing reasons at the gateway, or mostly request-level metrics?
Kartik N V J K
AI Developer | Making AI reliable, trustworthy & accessible to everyone | Active community contributor
Gateway-level observability is underrated, since the gateway is the one place you see every model call, retry, and fallback in one stream. What made mine useful was logging the decision, which model, why, and what it cost, not just latency and status. Are you capturing routing reasons at the gateway, or mostly request-level metrics?