A one-line prompt change refunded 212 customers twice in 38 minutes. Our offline eval had passed it at 96%. The line was: "If a tool call fails, retry once." In the eval, the mocked refund tool answer

Markdown supported · ⌘↵ to post
No responses yet.