---
title: "Discussion on \"I Built a 9× Faster LLM Memory Allocator in C++ — and Worked Out When It Pays Off\" | Hashnode"
description: "Discussion on \"I Built a 9× Faster LLM Memory Allocator in C++ — and Worked Out When It Pays Off\". A paged KV-cache allocator for Python, with every result checked against predictions I wrote before coding"
url: "https://hashnode.com/posts/i-built-a-9-faster-llm-memory-allocator-in-c-and-worked-out-when-it-pays-off/6ab6065e7b4232acdc17b980"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [I Built a 9× Faster LLM Memory Allocator in C++ — and Worked Out When It Pays Off](https://seng-wei-chieh.hashnode.dev/i-built-a-9-faster-llm-memory-allocator-in-c-and-worked-out-when-it-pays-off?utm_source=hashnode&utm_medium=feed)

A paged KV-cache allocator for Python, with every result checked against predictions I wrote before coding

Sep 25 · 18 min read · seng-wei-chieh.hashnode.dev  
Read the article

-   [#kv-cache](https://hashnode.com/tag/kv-cache)
-   [#vllm](https://hashnode.com/tag/vllm)
-   [#cpp](https://hashnode.com/tag/cpp)
-   [#performance-engineering](https://hashnode.com/tag/performance-engineering)
-   [#llm-inference](https://hashnode.com/tag/llm-inference)

## Written by

[schieh710](https://hashnode.com/@swchieh)

@swchieh · San Jose, USA

  
[View profile](https://hashnode.com/@swchieh)

AI/ML Infrastructure | System Design | MLOps

Infrastructure engineer based in the Bay Area specializing in the intersection of systems programming and machine learning. I’m focused on building scalable backends and high-performance MLOps pipelines, with a deep interest in CUDA-accelerated workloads and distributed systems design.

## Responses

Markdown supported · ⌘↵ to post

No responses yet.