TSTanisha Soniintanisha-soni.hashnode.dev·Jul 5 · 9 min readI Taught My Local Model to Speak LangChainTwo articles in, and both of them assumed something: that you're using a model LangChain already knows how to talk to. OpenAI, Anthropic, whatever, doesn't matter. LangChain has an adapter for it. Wha00
AKAlexander Kerchuminblog.kerchum.dev·Jun 5 · 12 min readLessons From the Bottom of the Stack: Shipping a QuantThe SCLP compression algorithm — palette the exponents, sidecar the outliers, pack the rest — was a week or so of prototyping. The two posts before this one covered it end to end. This post is about t00
PYPhaneesh Yalavarthiinphaneesh.hashnode.dev·May 11 · 1 min readLocal AI: Running Gemma 4 with llama.cpp and DockerIntroduction In the rapidly evolving landscape of Artificial Intelligence, the ability to run Large Language Models (LLMs) locally has become a game-changer for developers and researchers alike. Wheth00
강강문규indevsnack.hashnode.dev·May 6 · 10 min readQwen3.6 on DGX Spark: vLLM + NVFP4 + DFlash vs llama.cpp — 2x Faster at 88–104 tok/sTL;DR — I was happily running Qwen3.6 on llama.cpp. Then I saw claims of 2× speed with vLLM + NVFP4 + DFlash. So I installed it, fought through crashes, and measured it myself. Verdict: it's real. 88–00
PTPraneeth TSSinptss.hashnode.dev·May 4 · 8 min readWhy I built bonfire — a local-first terminal coding assistantI didn't start with a grand vision about privacy or open source. I just wanted something that ran on my machine. And didn't require me to upload my code to do it. On top of it, I was tired of my token10
VBVlad Butacuinomniforge.online·Apr 15 · 8 min readYour Local LLM Is Slow Because of Five Config FlagsYour model fits in memory. You load it up, send a prompt, and watch it choke halfway through a conversation. Or it runs, but at 3 tokens per second on hardware that should do better. You picked the ri00
NKNishchay Kaushikinnkaushik.in·Feb 13 · 5 min readHow to Run LLM Models on Old Android Devices LocallyThis post covers a much more technical and involving way to run local LLMs on android via a terminal setup. If you want to try an easy-to-use , less technical way, I have it covered in this latest bl00
IAIbrar Ansariinibraransari.hashnode.dev·Jan 26 · 7 min read🚀 7 Ways to Run Any LLMs Locally - Simple MethodsA comprehensive guide to running Large Language Models (LLMs) locally on your machine using various tools and platforms. 🎬 Video Demonstration 1. 🦙 Ollama - The Dominant Local LLM Ecosystem Ollama is the dominant ecosystem for running LLMs such a...00
AAmegillainsparktastic.hashnode.dev·Jan 4 · 2 min readUpdating GGUF Registry for new Llama Server UINo sooner had I build a GGUF model registry than llama.cpp released functionality to dynamically load and unload models from their new llama-server web UI! I had a play with this and realised that it doesn’t exactly work for my setup, mainly because ...00
AAmegillainsparktastic.hashnode.dev·Jan 3 · 6 min readPractical local LLM examples on DGX SparkI’ve had my Spark for a couple of months now. Since it is my only personal computer I’ve gone through the process of working out how to use local LLMs for general tasks like taking & editing notes, browsing the web and coding. This is what I’ve learn...00
