Good breakdown — the "measure each component's cost before optimizing" framing is the part most teams skip. One gotcha worth adding to the prompt caching section: cache entries are prefix-sensitive at the byte level. A system prompt with a timestamp, rotating user name, or even a trailing newline change invalidates the cached prefix silently, and the bill shows up as full-price input tokens with no error anywhere. Worth logging the cached-token field from the response per request — if the hit rate is zero on requests you believe share a prefix, that's the first place to look.
Seven
Good breakdown — the "measure each component's cost before optimizing" framing is the part most teams skip. One gotcha worth adding to the prompt caching section: cache entries are prefix-sensitive at the byte level. A system prompt with a timestamp, rotating user name, or even a trailing newline change invalidates the cached prefix silently, and the bill shows up as full-price input tokens with no error anywhere. Worth logging the cached-token field from the response per request — if the hit rate is zero on requests you believe share a prefix, that's the first place to look.