I should tighten my earlier wording: an all-green suite is useful, and the harness should still require green results for the behavior we already promise. The unresolved cases belong in a separate improvement set, with explicit expected behavior, rather than becoming failures everyone learns to ignore.
Then a prompt change can be checked against both questions: did it preserve the supported behavior, and did it resolve any known limitations? Once a limitation is fixed, promote that case into the regression suite. That keeps the signal without normalizing broken builds.
Worth adding the operational half to this: a prompt is a production artifact, so it needs the same handling as code. Version it in the repo rather than editing it in a console, and keep a small fixed set of examples you rerun after every edit - otherwise the only way anyone discovers a prompt change was bad is a user complaint two weeks later. The smart-contract example makes another point implicitly. When the output feeds code rather than a person, the highest-leverage instruction is usually the output contract: give it a schema and validate against it, so a malformed response fails loudly at the boundary instead of flowing downstream as a plausible-looking value.