The split between programmatic and LLM judges maps well to what I have found running agent evals. The boring checks (format, tool usage, file boundaries) are the ones that generate the most noise when done by a model, precisely because they are simple enough that a human would never disagree with the deterministic version. The real value of a hybrid approach is not just cost savings, but signal quality. When a program judge flags a format violation, the debugging step is zero. When an LLM judge flags the same thing, you have to read a paragraph of explanation, decide whether you trust it, and then check the raw output anyway. That subtle time tax adds up fast across hundreds of eval runs. The boundary I would draw is asymmetric: let the program judge be the arbiter of the checks it can express, and use the model judge only for the cases where no programmatic check exists yet. That way the model judge becomes a gap-filler rather than a primary evaluator, which makes its occasional mistakes easier to tolerate because the deterministic layer already caught the clear failures.