FeedDiscussion
Farhana Yasmin
Software Engineer
The false green section hits. We had a retry-based setup mask a broken coupon flow for 11 days because the test kept passing on the second attempt. Nobody looks at a green suite. Support tickets found the bug before CI did. Since then I trust a loud failure over a quiet pass every time.
The vendor question at the end is the useful bit. I asked our current tool's support team almost exactly this last quarter and got a healing percentage back, which tells you something. One thing the post doesn't cover: what does a redesign actually cost under intent re-resolution? If 200 cached steps invalidate at once, that's 200 AI resolutions in one run. Is that minutes or hours, and who pays for the tokens?
Good taxonomy, but I'd argue intent re-resolution doesn't eliminate brittleness, it relocates it. A selector is brittle against the DOM. "Add the item to the cart" is brittle against ambiguity. On a page with a quick-add button, a wishlist icon, and a buy-now CTA, the model is making a judgment call every time it re-resolves. The dual-model assertion helps, but two models agreeing is still probabilistic, not deterministic. Curious how you handle steps where the intent itself is ambiguous.