That silent fallback is a nasty one. A parse failure that returns a middle score looks like a result. I'd make the parser fail loudly and track the parse-failure rate as its own metric.

For the judge, I'd do both. Re-check it against a small human-labelled set on a fixed cadence, and also whenever the judge model or its prompt changes. Pinning the judge version helps, because then drift only happens when you choose it. A sudden shift in the score distribution with no system change is a good trigger too.