Tag feed

#llamacpp

23 posts3 followers

Explore Hashnode

Alternatives

Trending tags this week

AKAlexander Kerchumblog.kerchum.devJun 5 · 12 min read

Lessons From the Bottom of the Stack: Shipping a Quant

The SCLP compression algorithm — palette the exponents, sidecar the outliers, pack the rest — was a week or so of prototyping. The two posts before this one covered it end to end. This post is about t

0

PYPhaneesh Yalavarthiphaneesh.hashnode.devMay 11 · 1 min read

Local AI: Running Gemma 4 with llama.cpp and Docker

Introduction In the rapidly evolving landscape of Artificial Intelligence, the ability to run Large Language Models (LLMs) locally has become a game-changer for developers and researchers alike. Wheth

0

강강문규devsnack.hashnode.devMay 6 · 10 min read

Qwen3.6 on DGX Spark: vLLM + NVFP4 + DFlash vs llama.cpp — 2x Faster at 88–104 tok/s

TL;DR — I was happily running Qwen3.6 on llama.cpp. Then I saw claims of 2× speed with vLLM + NVFP4 + DFlash. So I installed it, fought through crashes, and measured it myself. Verdict: it's real. 88–

0

PTPraneeth TSSptss.hashnode.devMay 4 · 8 min read

Why I built bonfire — a local-first terminal coding assistant

I didn't start with a grand vision about privacy or open source. I just wanted something that ran on my machine. And didn't require me to upload my code to do it. On top of it, I was tired of my token

0

VBVlad Butacuomniforge.onlineApr 15 · 8 min read

Your Local LLM Is Slow Because of Five Config Flags

Your model fits in memory. You load it up, send a prompt, and watch it choke halfway through a conversation. Or it runs, but at 3 tokens per second on hardware that should do better. You picked the ri

0

NKNishchay Kaushiknkaushik.inFeb 13 · 5 min read

How to Run LLM Models on Old Android Devices Locally

This post covers a much more technical and involving way to run local LLMs on android via a terminal setup. If you want to try an easy-to-use , less technical way, I have it covered in this latest bl

0

IAIbrar Ansariibraransari.hashnode.devJan 26 · 7 min read

🚀 7 Ways to Run Any LLMs Locally - Simple Methods

A comprehensive guide to running Large Language Models (LLMs) locally on your machine using various tools and platforms. 🎬 Video Demonstration 1. 🦙 Ollama - The Dominant Local LLM Ecosystem Ollama is the dominant ecosystem for running LLMs such a...

0

AAmegillasparktastic.hashnode.devJan 4 · 2 min read

Updating GGUF Registry for new Llama Server UI

No sooner had I build a GGUF model registry than llama.cpp released functionality to dynamically load and unload models from their new llama-server web UI! I had a play with this and realised that it doesn’t exactly work for my setup, mainly because ...

0

AAmegillasparktastic.hashnode.devJan 3 · 6 min read

Practical local LLM examples on DGX Spark

I’ve had my Spark for a couple of months now. Since it is my only personal computer I’ve gone through the process of working out how to use local LLMs for general tasks like taking & editing notes, browsing the web and coding. This is what I’ve learn...

0

AAmegillasparktastic.hashnode.devDec 27, 2025 · 5 min read

Creating a GGUF Registry for local models on my DGX Spark

Prior to acquiring a DGX Spark, my experience running local LLMs was limited to basic experimentation with Ollama. So when I got my new toy I went a bit crazy downloading lots of different models. And then ended up with a page of notes containing a l...

0

#llamacpp

Search Hashnode

#llamacpp

Explore Hashnode

Trending tags this week

Lessons From the Bottom of the Stack: Shipping a Quant

Local AI: Running Gemma 4 with llama.cpp and Docker

Qwen3.6 on DGX Spark: vLLM + NVFP4 + DFlash vs llama.cpp — 2x Faster at 88–104 tok/s

Why I built bonfire — a local-first terminal coding assistant

Your Local LLM Is Slow Because of Five Config Flags

How to Run LLM Models on Old Android Devices Locally

🚀 7 Ways to Run Any LLMs Locally - Simple Methods

Updating GGUF Registry for new Llama Server UI

Practical local LLM examples on DGX Spark

Creating a GGUF Registry for local models on my DGX Spark