---
title: "Tao Pu (@putao520) | Hashnode"
description: "Tao Pu (@putao520) writes on Hashnode. Read their latest articles and follow their work."
url: "https://hashnode.com/@putao520"
---

@putao520

# Tao Pu

@putao520 · Joined February 2026

### [Building an AI Inference Gateway: Practice and Insights](https://www.putao282.com/building-an-ai-inference-gateway-practice-and-insights?utm_source=hashnode&utm_medium=feed)

\# Building an AI Inference Gateway: Practice and Insights Recently, while building an AI inference gateway, I encountered quite a few challenges. AI gateways differ significantly from traditional API gateways: request bodies can be tens of KB, infer...

15 min read · [putao282.com](https://www.putao282.com/building-an-ai-inference-gateway-practice-and-insights?utm_source=hashnode&utm_medium=feed)

[Feb 4](https://hashnode.com/posts/building-an-ai-inference-gateway-practice-and-insights/698306a82c51d72a2e7c79fa)

### [Building an AI Inference Gateway, Part 2: Unifying Streaming and Non-Streaming Processing](https://www.putao282.com/building-an-ai-inference-gateway-part-2-unifying-streaming-and-non-streaming-processing?utm_source=hashnode&utm_medium=feed)

\# Building an AI Inference Gateway, Part 2: Unifying Streaming and Non-Streaming Processing Recently, while developing \[llm-converter\](https://github.com/putao520/llm-converter)—a Rust library for LLM protocol conversion—I encountered a problem: \*\*h...

5 min read · [putao282.com](https://www.putao282.com/building-an-ai-inference-gateway-part-2-unifying-streaming-and-non-streaming-processing?utm_source=hashnode&utm_medium=feed)

[Feb 4](https://hashnode.com/posts/building-an-ai-inference-gateway-part-2-unifying-streaming-and-non-streaming-processing/6983042c62c0ca9c41a7faf3)
