Thanks Kartik. You got Clause 4 exactly: the veto only works if nothing is allowed to average it away.

On the corpus, I'd be guessing if I gave you a number, and the post doesn't give one either. What I watch is what's in the set, not how big it is. It's hand-labeled and versioned, and I seed it from two places: every case the judge got wrong before, and the borderline band. Clear passes almost never teach you anything. A judge change passes when nothing flipped between fails and clears, and I can explain each borderline move. One fails-to-clears flip is a regression even on a small set, and a huge set made of easy passes proves nothing.

It also keeps growing. Each time the judge and a human disagree in production, that case goes into the set, so it grows where the judge was weakest.