The disorder problem is the sharp edge in practice — most idempotency writeups only cover duplicate delivery, not out-of-order delivery, and those need different fixes. A dedup key stops the double-credit, but if a 'captured' webhook arrives after a 'refunded' one because of a retry queue, a naive last-write-wins handler still corrupts state. What held up for us was making every webhook carry a monotonic sequence per transaction and rejecting (not just deduping) events older than the current recorded state — cheap to add, easy to miss until a retry storm exposes it in prod.
Henry Robt
The section on idempotency and reconciliation is especially important for payment webhooks, where duplicate or delayed events can easily cause inconsistent transaction states. Having a clear process for handling retries and reconciling payment records can make integrations much more reliable.