The Failure of Legacy Evaluation in AI

The history of natural language processing evaluation reveals a persistent pattern. Metrics created for one generation of technology become dangerously inadequate for the next. When statistical machine translation systems dominated the field, BLEU sc...