The three questions are the right frame, especially the second one. In my experience the double run is rarely a scheduler bug, it is two workers both thinking they own the job. I now treat every scheduled job as if it will run twice, so I give it a unique key or a lock and make the work safe to repeat. The other change that helped me most was alerting when the expected run is missing, not just when a run fails, because a silent miss is the one nobody notices until a customer does.