Great teardown. The size gap you documented is real on the ground: when I needed a model for a fully offline tool (explaining official letters, the kind of text that cannot leave the machine), nothing in the newest families was small enough for plain CPU, so I ended up on a 1B Gemma. Slower than your L4 numbers, about 40 seconds per document end to end, but the task budget allowed it. Your memory math is the part most "run it locally" tutorials skip: 18 GB before context is not a laptop model for most people. Did you measure how much extra memory the context window adds as you raise it past 4,096?
Great teardown. The size gap you documented is real on the ground: when I needed a model for a fully offline tool (explaining official letters, the kind of text that cannot leave the machine), nothing in the newest families was small enough for plain CPU, so I ended up on a 1B Gemma. Slower than your L4 numbers, about 40 seconds per document end to end, but the task budget allowed it. Your memory math is the part most "run it locally" tutorials skip: 18 GB before context is not a laptop model for most people. Did you measure how much extra memory the context window adds as you raise it past 4,096?