Mmamonuinmamonu.hashnode.dev·Sep 12 · 14 min readWhat Fits and What WorksThe instinct, when you have a small card, is to go shopping for the smallest possible version of the biggest possible model. It is a fun problem. There is a whole ecosystem of increasingly heroic comp00
Mmamonuinmamonu.hashnode.dev·Sep 10 · 9 min readTrusting Local LLMs with Code: Building little-llms-benchWhen it comes to code generation, looks can be deceiving. As developers, we need to know whether the small, self-hosted LLMs running on our local setups can actually produce functional code, or if the00
Mmamonuinmamonu.hashnode.dev·Sep 10 · 15 min readRunning Large MoE LLMs on Modest Hardware with FreeTokenOpen-weight AI models have become seriously capable. The slightly awkward part is that some of them are also enormous. Running a large model locally normally means having enough GPU memory to hold it,00
Mmamonuinmamonu.hashnode.dev·Sep 9 · 24 min readUsing a mac mini (2012) for LLM inference with KoboldCpp … especially when your inference server is a potato. I have a 2012 Intel Mac mini. It is one of those machines that still works surprisingly well, provided you don't keep asking it to behave like some00
Mmamonuinmamonu.hashnode.dev·Sep 3 · 5 min readTesting using Property Based TestsQuis custodiet ipsos custodes? is a Latin phrase found in the work of the Roman poet Juvenal in his Satires (Satire VI, lines 347–348). It translates to “but who will guard the guardians?”—or, in deve01N
