PLPujun Luninpujun-ml-sys-blogs.hashnode.dev·4h ago · 34 min readWhen Does Heterogeneous Inference Pay Off?Apple Silicon’s shared memory made CPU/GPU disaggregation look easier. While building mini-vllm-rs, a single-node LLM inference engine for macOS, I considered prefill–decode disaggregation (P/D) and t00
PLPujun Luninpujun-ml-sys-blogs.hashnode.dev·6d ago · 19 min readBuilding an LLM Inference Engine with AI: The Code Was the Easy PartTL;DR This post is about what systems engineering looks like when AI makes implementation much cheaper, but understanding the system, choosing the right abstractions, and reasoning about performance r00
