1d ago · 5 min read · Originally published at https://nicoloboschi.com/posts/20260402 TL;DR: Memory benchmarks died when context windows hit 1M tokens — just dump everything in the prompt. BEAM tests at 10M where that trick fails. Hindsight scores 64.1% there, 58% ahead ...
Join discussion5d ago · 6 min read · Alibaba just dropped four models and said "here, they're free." The Qwen 3.5 Small family — 0.8B, 2B, 4B, and 9B parameter models — is fully open source under Apache 2.0. No gated access, no usage restrictions, no phone-home telemetry. Download the w...
Join discussion
Mar 26 · 6 min read · Originally published at https://nicoloboschi.com/posts/20260324 TL;DR Supermemory published a "~99% SOTA" memory benchmark result that was actually a stunt - and the backlash was immediate and deserved. Gaming benchmarks erodes trust in an industry ...
Join discussionMar 23 · 7 min read · Spry Performance Benchmarks: Measuring and Optimizing Dart Server Performance Performance is critical for production‑ready applications. In this comprehensive guide, we'll explore how to benchmark, measure, and optimize Spry applications for maximum ...
Join discussionMar 15 · 2 min read · The AI safety community has built an evaluation apparatus specifically to detect dangerous capabilities before deployment. The premise is clear: test the model before you ship it. If it can help synthesize pathogens or scheme against its operators, c...
Join discussionMar 8 · 9 min read · Three months ago my team lead asked me to pick one AI coding tool for our five-person team to standardize on. We're a fintech startup — TypeScript on the frontend, Django on the backend, a fair amount of gnarly financial calculation logic. We couldn'...
Join discussionFeb 26 · 3 min read · The Car Wash Test Opper tested 53 AI models with a dead-simple question: "I want to wash my car. The car wash is 50 meters away. Should I walk or drive?" The answer is obvious: drive. The car needs to be at the car wash. Out of 53 models, 42 said w...
Join discussionFeb 5 · 4 min read · I expected the 30B model to be the better judge. It wasn't. When I set out to replace OpenAI's GPT-4o-mini as the judge for the Oolong benchmark, my plan was simple: use the biggest local model I had. Qwen3-coder at 30B parameters seemed like the obv...
Join discussionFeb 1 · 1 min read · What is SWE-bench? SWE-bench is a widely followed benchmark evaluation framework designed to test AI coding assistants on real software engineering tasks. AI coding assistant benchmarks are supposed to give us clarity. SWE-bench does the opposite. SW...
Join discussion
