Passing down what we know. Discovering what's next.
Father of two, tech lover. Building systems by day, raising curious minds by night. I write hands-on guides and reproducible experiments on running, fine-tuning and evaluating AI locally on Apple Silicon, on AI agents, and on how AI changes software and finance. The code behind every experiment is on GitHub: github.com/hslimi/night-thoughts
Markdown supported · ⌘↵ to post
The bigger-is-smarter rule breaking is what makes frontier-model news matter less than efficiency per task. Smaller models with better data and test-time compute are already closing the gap on many real workloads, while the largest runs get harder to justify on cost. Do you think the next real jump comes from new architectures, or from smarter inference-time reasoning on models of this size? iin1005h0628
Your distinction between an overnight background task and an interactive chatbot makes the parameter-count comparison much more actionable. One extra constraint belongs in that decision: whether the model can finish the entire workload within the deadline, including retries and long inputs, rather than merely produce an acceptable answer in isolation.
I would compare candidates on completed tasks per available window, with the same input set and an explicit failure policy. A smaller model that needs several repair attempts can lose its apparent speed advantage. Conversely, a slower model may be entirely suitable when its first answer reliably completes the job.
Kartik N V J K
Your line that a model topping one leaderboard can look mediocre on another matches my experience: a coding agent I picked off a leaderboard flopped on our repo. I make the final call by replaying our own merged work, as I wrote up here: kartiknvjk.hashnode.dev/i-picked-a-coding-agent-o…. Do you have numbers for the 70B model's quality loss on that 4GB card?