Your AI Agent Improved Its Score. Did It Improve the System?

Imagine opening the results from a thousand overnight AI experiments. One claims to have made your research agent much better at the benchmark you use to choose its next version. You open the diff. Al