ARAI ROUTERinai-router.hashnode.dev·Aug 4 · 18 min readDo Not Retry This .NET Request Yet: OperationCanceledException, 429, Retry-After, and Partial StreamsAn OpenAI-compatible request can fail in a way that looks transient while leaving the client unable to prove that automatic replay is safe. The most common examples are: OperationCanceledException, w00
ARAI ROUTERinai-router.hashnode.dev·Aug 4 · 12 min readA 200 from /models Is Not an OpenAI-Compatible API Smoke TestYour new API base URL resolves. TLS succeeds. GET /models returns HTTP 200. That is useful evidence. It is not proof that the endpoint is ready for your application. The response might have the wrong 00
ARAI ROUTERinai-router.hashnode.dev·Jul 28 · 9 min readNo Tokens Yet Does Not Mean a Rust LLM Stream Is Safe to RetryYour LLM request returned HTTP 200. The stream produced no text. Then the connection reset. Is it safe to retry the whole request? Not necessarily. “No text reached the UI” answers only one narrow que00
ARAI ROUTERinai-router.hashnode.dev·Jul 28 · 7 min readTest openai-python Retries with HTTPX MockTransportTo measure how many requests openai-python really sends, inject an httpx.Client backed by MockTransport, count handler calls, and return a synthetic 429. With openai==2.49.0, the default max_retries=200
ARAI ROUTERinai-router.hashnode.dev·Jul 5 · 5 min readConfiguring an OpenAI-compatible base URL in Python and Node.jsMany AI applications start with the official OpenAI SDK pattern: create a client, pass an API key, send a chat completion request, and read the response. That pattern is useful because a lot of compat00
