Upgrading the judge ends one score series and starts another

There is a mature literature on what happens when you swap one measuring instrument for another, and it is not in machine learning. The standard treatment is Bland and Altman, "Statistical methods for