The interesting part here is that the harness makes agent infrastructure portable, but the live-testing section shows why portability alone isn't enough. Once context handling, reasoning replay, caching, and model-specific behavior become runtime assumptions, those assumptions need to be characterized rather than buried in configuration.

I especially like keeping the hook even after the documented failure couldn't be reproduced. That’s a good production pattern: distinguish “currently reproducible” from “still worth guarding against,” and encode the expected behavior in a characterization test. Otherwise a future Bedrock/model revision can silently invalidate an assumption that nobody remembers was there.

For long-running agents, I’d treat those characterization tests almost like compatibility contracts between the harness and model provider. The model string may be swappable, but the behavioral contract around context, tool calls, reasoning state, and session replay is what actually determines whether the agent remains reliable.