Awaiting drain bounds the application's writes, but pausing iteration does not necessarily pause the remote model's generation. Tokens can still accumulate in an SDK queue, proxy buffer or transport layer depending on how that upstream stream is implemented.
I would test the complete path with a deliberately slow reader and a long completion, measuring resident memory and buffered bytes beyond the response object. A maximum stall duration can provide a clear cancellation policy when the client never catches up. That would validate the memory bound end to end rather than assuming the res.write return value controls every queue between the model and browser.
Ahmet Özel
AI Engineer. Computer Vision, RAG and LLM agents.
Awaiting drain bounds the application's writes, but pausing iteration does not necessarily pause the remote model's generation. Tokens can still accumulate in an SDK queue, proxy buffer or transport layer depending on how that upstream stream is implemented.
I would test the complete path with a deliberately slow reader and a long completion, measuring resident memory and buffered bytes beyond the response object. A maximum stall duration can provide a clear cancellation policy when the client never catches up. That would validate the memory bound end to end rather than assuming the res.write return value controls every queue between the model and browser.