--- title: "#llama-cpp articles | Hashnode" description: "Explore 16 articles tagged #llama-cpp on Hashnode. Tutorials, guides, and how-tos from the community." url: "https://hashnode.com/tag/llama-cpp" --- Tag # #llama-cpp 16 posts · 0 followers [Limin Ma](https://hashnode.com/@liminma) · [Oct 24](https://hashnode.com/posts/where-confidence-comes-from/6adbf59943a562bc0e8da4a6) ### [Where confidence comes from](https://liminma.hashnode.dev/where-confidence-comes-from?utm_source=hashnode&utm_medium=feed) Reading how sure an LLM was of its answer, from the probabilities it already computed 14 min read · [liminma.hashnode.dev](https://liminma.hashnode.dev/where-confidence-comes-from?utm_source=hashnode&utm_medium=feed) [Limin Ma](https://hashnode.com/@liminma) · [Oct 18](https://hashnode.com/posts/forcing-a-valid-answer/6ad5554c1b96e44cbc96cd2d) ### [Forcing a valid answer](https://liminma.hashnode.dev/forcing-a-valid-answer?utm_source=hashnode&utm_medium=feed) Grammars, one-letter choices, and an LLM that can only answer with a label from your list 14 min read · [liminma.hashnode.dev](https://liminma.hashnode.dev/forcing-a-valid-answer?utm_source=hashnode&utm_medium=feed) [circuitrocks](https://hashnode.com/@circuitrocks) · [Oct 5](https://hashnode.com/posts/cantastorie-offline-ai-storytelling-toy-on-arduino-uno-q/6ac36a48575ccf237f574f5f) ### [CantaStorie: Offline AI Storytelling Toy on Arduino UNO Q](https://circuitrocks.hashnode.dev/cantastorie-offline-ai-storytelling-toy-on-arduino-uno-q?utm_source=hashnode&utm_medium=feed) CantaStorie Offline AI Storytelling: CantaStorie runs YOLOX-Nano and Gemma 1B on an Arduino UNO Q to spot an object and narrate a custom story, with zero… 2 min read · [circuitrocks.hashnode.dev](https://circuitrocks.hashnode.dev/cantastorie-offline-ai-storytelling-toy-on-arduino-uno-q?utm_source=hashnode&utm_medium=feed) [1](https://hashnode.com/posts/cantastorie-offline-ai-storytelling-toy-on-arduino-uno-q/6ac36a48575ccf237f574f5f) [Amaresh Pelleti](https://hashnode.com/@amareswer) · [Oct 1](https://hashnode.com/posts/ollama-vs-vllm-vs-llama-cpp-which-local-llm-engine/6abe517a5fc863159262d968) ### [Ollama vs vLLM vs llama.cpp: Which Local LLM Engine?](https://devtoolhub.hashnode.dev/ollama-vs-vllm-vs-llama-cpp-which-local-llm-engine?utm_source=hashnode&utm_medium=feed) Originally published on DevToolHub. Ollama vs vLLM vs llama.cpp comes down to one question: how many people will hit the model at once? For one developer on a laptop, pick Ollama or llama.cpp. For a 9 min read · [devtoolhub.hashnode.dev](https://devtoolhub.hashnode.dev/ollama-vs-vllm-vs-llama-cpp-which-local-llm-engine?utm_source=hashnode&utm_medium=feed) [2](https://hashnode.com/posts/ollama-vs-vllm-vs-llama-cpp-which-local-llm-engine/6abe517a5fc863159262d968) [Aman Karki](https://hashnode.com/@amankarki) · [Sep 27](https://hashnode.com/posts/llama-cpp-two-cuda-prs/6ab9471561ca1c18dd7082e6) ### [Two CUDA PRs into llama.cpp, and what they taught me about a 100K-star codebase](https://amankarki.hashnode.dev/llama-cpp-two-cuda-prs?utm_source=hashnode&utm_medium=feed) A 95-line pooling kernel, a 9-line fix for a silent CPU fallback, and how ggml decides where your ops run 11 min read · [amankarki.hashnode.dev](https://amankarki.hashnode.dev/llama-cpp-two-cuda-prs?utm_source=hashnode&utm_medium=feed) [Mohammad Hatif Osmani](https://hashnode.com/@mhosm) · [Sep 7](https://hashnode.com/posts/building-llamini-cpp-a-from-scratch-llama-cpp-core-in-c99-zero-dependencies/6a9f17866f526dd1c8ea4868) ### [Building llamini.cpp: a from-scratch llama.cpp core in C99, zero dependencies](https://mhosm.hashnode.dev/building-llamini-cpp-a-from-scratch-llama-cpp-core-in-c99-zero-dependencies?utm_source=hashnode&utm_medium=feed) At a glance Dialect Pure C99 (POSIX + libc) — 0 third-party dependencies Build One command: make → llamini Models TinyLlama 1.1B, Qwen2.5 0.5B, GPT-2 124M, Gemma 2B Quants Q4\_K, Q6\_K, Q5 26 min read · [mhosm.hashnode.dev](https://mhosm.hashnode.dev/building-llamini-cpp-a-from-scratch-llama-cpp-core-in-c99-zero-dependencies?utm_source=hashnode&utm_medium=feed) [Brian King](https://hashnode.com/@SoloDevNZ) · [Jul 9](https://hashnode.com/posts/installing-rocm-llama-cpp-for-cpus-apus/6a4ff91051507adef83436fc) ### [Installing ROCm & Llama.cpp for CPUs & APUs.](https://solodev.app/installing-rocm-llama-cpp-for-cpus-apus?utm_source=hashnode&utm_medium=feed) Or: Adding a Device Driver for AMD GPUs & iGPUs. 9 min read · [solodev.app](https://solodev.app/installing-rocm-llama-cpp-for-cpus-apus?utm_source=hashnode&utm_medium=feed) [Hemkesh](https://hashnode.com/@hemkesh) · [Jun 17](https://hashnode.com/posts/why-cudamalloc-fails-on-nvidia-jetson-orin-nano-super-and-the-one-flag-that-fixes-it/6a32ada98db3d147576cdde4) ### [Why cudaMalloc fails on NVIDIA Jetson Orin Nano Super — and the one flag that fixes it](https://hemkesh.hashnode.dev/why-cudamalloc-fails-on-nvidia-jetson-orin-nano-super-and-the-one-flag-that-fixes-it?utm_source=hashnode&utm_medium=feed) A unified-memory allocator fix for running llama.cpp on Jetson's shared GPU/CPU memory pool. 2 min read · [hemkesh.hashnode.dev](https://hemkesh.hashnode.dev/why-cudamalloc-fails-on-nvidia-jetson-orin-nano-super-and-the-one-flag-that-fixes-it?utm_source=hashnode&utm_medium=feed) [Phaneesh Yalavarthi](https://hashnode.com/@Phaneesh) · [May 11](https://hashnode.com/posts/local-ai-running-gemma-4-with-llama-cpp-and-docker/6a01a188fca21b0d4bf5aed6) ### [Local AI: Running Gemma 4 with llama.cpp and Docker](https://phaneesh.hashnode.dev/local-ai-running-gemma-4-with-llama-cpp-and-docker?utm_source=hashnode&utm_medium=feed) Local AI: Running Gemma 4 with llama.cpp and Docker 1 min read · [phaneesh.hashnode.dev](https://phaneesh.hashnode.dev/local-ai-running-gemma-4-with-llama-cpp-and-docker?utm_source=hashnode&utm_medium=feed) [강문규](https://hashnode.com/@devsnack) · [May 6](https://hashnode.com/posts/qwen3-6-on-dgx-spark-vllm-nvfp4-dflash-vs-llama-cpp-2x-faster-at-88-104-tok-s/69fb2aa950ecad45331b9b3a) ### [Qwen3.6 on DGX Spark: vLLM + NVFP4 + DFlash vs llama.cpp — 2x Faster at 88–104 tok/s](https://devsnack.hashnode.dev/qwen3-6-on-dgx-spark-vllm-nvfp4-dflash-vs-llama-cpp-2x-faster-at-88-104-tok-s?utm_source=hashnode&utm_medium=feed) I benchmarked Qwen3.6-35B-A3B on DGX Spark GB10 using llama.cpp (45–50 tok/s) vs vLLM + NVFP4 + DFlash (88–104 tok/s). Full setup guide, docker-compose, troubleshooting, and real-world results. 10 min read · [devsnack.hashnode.dev](https://devsnack.hashnode.dev/qwen3-6-on-dgx-spark-vllm-nvfp4-dflash-vs-llama-cpp-2x-faster-at-88-104-tok-s?utm_source=hashnode&utm_medium=feed)