---
title: "Comment by Yunus Emre Vurgun on \"Using a mac mini (2012) for LLM inference with KoboldCpp \" | Hashnode"
description: "The finding that architecture beat tuning is the useful one. A reason Ling-style MoE models fit this box so well is that decoding on an old CPU is limited by me"
url: "https://hashnode.com/posts/using-a-mac-mini-2012-for-llm-inference-with-koboldcpp/6aa184fbdb8661a7a6a47a26/comment/6ad7280556de4f21ca8a7ad4"
---

[Discussion](https://hashnode.com/posts/using-a-mac-mini-2012-for-llm-inference-with-koboldcpp/6aa184fbdb8661a7a6a47a26)

Comment [Using a mac mini (2012) for LLM inference with KoboldCpp](https://hashnode.com/posts/using-a-mac-mini-2012-for-llm-inference-with-koboldcpp/6aa184fbdb8661a7a6a47a26)

[Yunus Emre Vurgun](https://hashnode.com/@yunusemrevurgun)

[Oct 20](https://hashnode.com/posts/using-a-mac-mini-2012-for-llm-inference-with-koboldcpp/6aa184fbdb8661a7a6a47a26/comment/6ad7280556de4f21ca8a7ad4)

The finding that architecture beat tuning is the useful one. A reason Ling-style MoE models fit this box so well is that decoding on an old CPU is limited by memory bandwidth, and a MoE only reads the active experts per token, so it moves far less data than a dense 7B. For the growing-context problem, one thing to try is a quantized KV cache only if your build supports it without the slowdown you saw, otherwise keeping the system prompt fixed lets prompt caching skip re-processing it each request. Also worth checking that the 2012 Mini isn't thermal throttling during long runs, since a sustained tokens/sec drop is easy to mistake for a context effect.