When Your Application Can’t Explain Its Own State

Some of the hardest production incidents are not full outages. They are the incidents where an operation finished in one place and not another. A payment provider accepts a charge, but the HTTP client