The point that each use case needs its own definition of an acceptable result is where I've watched eval efforts stall, because teams reach for generic metrics first. For summaries I check whether anything appears that is absent from the source, the same criterion you list, and that one check catches more for me than any fluency score. How do you decide which cases to rerun before a prompt change ships?