Nothing here yet.
Good point - I'll admit I was thinking of the eval set mostly as a regression check, but you're right that an all green suite eventually just confirms the harness works, not that the prompt is good. Deliberately keeping a few should fail cases flips it into an actual signal. Worth building into how I set these up going forward.
Absolutely. Treating prompts as production artifacts is an important part of making prompt engineering reliable. Versioning prompts and rerunning a fixed evaluation set after every change gives teams a much better way to catch regressions before users do. I also agree on the output contract. When LLM output feeds another system or code path, schema validation creates a clear boundary where malformed responses can fail loudly instead of becoming downstream issues. Great point!
