A 4% cache hit rate was costing us money. Here is the arithmetic I should have run first.

We turned on prompt caching for our document-QA service and the invoice went up. Not dramatically. About 5%. Enough that I assumed it was traffic growth for the first two weeks, and it was not. This p