The origin can't always be the loud one, and your GitLab example shows why: the backup job did try to report, and the mail server rejected its failure emails. A broken component often can't say it's broken. So I'd pair loud-at-origin with a check that expects good news and alarms when it doesn't arrive, run by something other than the job itself. A rule like "no backup above a size floor in the last 26 hours" would have fired there without the backup job having to work.
The fallback path needs the same treatment from the caller's side. The caller is the only place that can count how often it fell back, so it should publish that rate as a number with a named owner and a ceiling, rather than page on it. A cache serving stale answers 2% of the time is probably fine; 40% for a week means the real path is gone, and the trend shows that long before the day it breaks.