Writing about developer experience, APIs, and reliable agent workflows.
I build AgentSEO.dev and write about developer tools, APIs, and reliable agent workflows.
Markdown supported · ⌘↵ to post
The distinction between a resolved call and a successful operation is easy to miss. In addition to checking isError, a client can validate a strict success schema and treat absent required fields or an unexpected error envelope as an integration failure, not as an empty successful result. It would be useful to add malformed JSON and a valid JSON object with the wrong shape to the fixture, and assert that neither reaches the next tool. Keeping protocol errors, flagged tool failures, and schema failures as separate telemetry categories would also make version changes easier to diagnose.
A flagged error from a tool that writes something still doesn't tell the agent whether the write happened. Your thrown_error row is the one I'd extend: if the handler throws "upstream timeout" after the upstream already committed, the client gets isError: true, and the obvious recovery is to call again, which repeats the write.
For side-effecting tools I'd put the effect state in the error text itself: not applied, applied, or unknown. Unknown is the honest answer for a timeout, and it should tell the agent to check status before retrying. If the tool also takes an idempotency key and the client reuses it on the retry, the server can return the first result instead of doing the work twice. That would make a sixth row for the fixture: a handler that commits and then throws, followed by a second call with the same key.
Kartik N V J K
I did not expect the unknown-tool row. The spec describes it as a JSON-RPC protocol error, and your SDK 1.26.0 fixture got a resolved result carrying isError true instead, so the split I was treating as reliable, protocol rejection versus tool failure, depends on the SDK version. I hit something similar chasing retry behavior across MCP servers, so a pinned version-specific test reads like the only safe answer.