Mmamonuinmamonu.hashnode.dev·16h ago · 14 min readWhat Fits and What WorksThe instinct, when you have a small card, is to go shopping for the smallest possible version of the biggest possible model. It is a fun problem. There is a whole ecosystem of increasingly heroic comp00
Mmamonuinmamonu.hashnode.dev·2d ago · 9 min readTrusting Local LLMs with Code: Building little-llms-benchWhen it comes to code generation, looks can be deceiving. As developers, we need to know whether the small, self-hosted LLMs running on our local setups can actually produce functional code, or if the00
Mmamonuinmamonu.hashnode.dev·2d ago · 15 min readRunning Large MoE LLMs on Modest Hardware with FreeTokenOpen-weight AI models have become seriously capable. The slightly awkward part is that some of them are also enormous. Running a large model locally normally means having enough GPU memory to hold it,00
Mmamonuinmamonu.hashnode.dev·3d ago · 9 min readUsing a mac mini (2012) for LLM inference with KoboldCpp I have an old by now 2012 Intel Mac Mini. Its one of these amazing machines that still work really well if you dont try to put any 2020-era software on it. So I have been making it a music server to a00
Mmamonuinmamonu.hashnode.dev·Sep 3 · 5 min readTesting using Property Based TestsQuis custodiet ipsos custodes? is a Latin phrase found in the work of the Roman poet Juvenal in his Satires (Satire VI, lines 347–348). It translates to “but who will guard the guardians?”—or, in deve00
