An experiment with GPT-6 Luna on 240 puzzles.
Been breaking AI since Y2K
I'm an AI scientist and engineer with over 20 years in the field, from a PhD in robotics and AI to building systems in pharma, quick commerce, and fintech. I like turning research ideas into things that work in production. On BreakingAI, I put AI tools and techniques through hands-on experiments and share what holds up and what breaks.
Markdown supported · ⌘↵ to post
Kartik N V J K
Dividing by 1.4736 when the rate required multiplication is the example that settled it for me, a perfect evaluation of the wrong operation. I expected low effort plus tools to be the budget answer, and your 2.15x token multiplier for 92% against 96% at plain high effort reads the opposite way. Did the nine puzzles the calculator flipped from correct to wrong share anything, like the extra tool turns burying an original constraint?