The double-charge example is the boundary that changes the conversation from “glue” to a real workflow system. My rule is that every side effect needs a business idempotency key owned by the caller—not just the provider’s request ID—and a durable record of intent before execution.
For CRM/voice workflows we use an outbox-style command record with: workflow ID, target system, normalized operation, idempotency key, attempt, and evidence pointer. The worker can retry indefinitely, but the target write is either naturally idempotent or guarded by that key. Compensation is explicit; it is never hidden in a catch block.
We learned this while building Atlantic after seeing retries create duplicate contact activity. Making the workflow state visible also gave operators a safe “replay from here” control instead of rerunning the whole chain.
I’m Ege, co-founder of Atlantic, sharing our production pattern. Your point about success and retry paths being authored months apart is exactly why the invariant has to live in shared workflow infrastructure.
The double-charge example is the boundary that changes the conversation from “glue” to a real workflow system. My rule is that every side effect needs a business idempotency key owned by the caller—not just the provider’s request ID—and a durable record of intent before execution.
For CRM/voice workflows we use an outbox-style command record with: workflow ID, target system, normalized operation, idempotency key, attempt, and evidence pointer. The worker can retry indefinitely, but the target write is either naturally idempotent or guarded by that key. Compensation is explicit; it is never hidden in a catch block.
We learned this while building Atlantic after seeing retries create duplicate contact activity. Making the workflow state visible also gave operators a safe “replay from here” control instead of rerunning the whole chain.
I’m Ege, co-founder of Atlantic, sharing our production pattern. Your point about success and retry paths being authored months apart is exactly why the invariant has to live in shared workflow infrastructure.