You are right that inspecting reasoning and tool calls still does not tell you the outcome was correct. I score the outcome against user intent, because an agent can take every step cleanly and still land on the wrong page. For browser agents, how are you defining a correct outcome in a way you can check automatically?
Exactly. I define success as observable postconditions tied to user intent, then verify them independently with URL and DOM checks plus test artifacts. The verifier should judge the result, not trust the agent’s reasoning.
Kartik N V J K
AI Developer | Making AI reliable, trustworthy & accessible to everyone | Active community contributor
You are right that inspecting reasoning and tool calls still does not tell you the outcome was correct. I score the outcome against user intent, because an agent can take every step cleanly and still land on the wrong page. For browser agents, how are you defining a correct outcome in a way you can check automatically?