Someone Else's Words in Your Answer: A Design Review of Cloudflare's GLM-5.3 KV Cache Exposure
A fluent reply with someone else's words in it Picture pasting a whole code archive into an AI coding model and asking where a bug lives. The answer streams back quickly and reads well, yet mixed into
Responses
Markdown supported · ⌘↵ to post
The detail that stayed with me is the status code. A refusal that still reaches decode and then returns a fluent 200 gives the user no signal that anything went wrong. The output reads well, so nobody questions whose words are in it. That is what makes a shared-cache bug different from an ordinary error.
Your framing of the two limits disagreeing is the core of it. One check says the request fits, the other says the pool is smaller, and the request is allowed to proceed anyway. In a shared system that is not a performance detail, it is a trust boundary. The refusal has to happen at the door, and pages should not be released back into the pool until the writers using them are done.
For anyone buying or using a shared AI service, the question this raises is practical: how is one customer's data isolated in shared inference, not just encrypted at rest? If a vendor cannot explain that clearly, the speed benefit of the shared cache is not worth the risk.