Building an LLM Inference Engine with AI: The Code Was the Easy Part

TL;DR This post is about what systems engineering looks like when AI makes implementation much cheaper, but understanding the system, choosing the right abstractions, and reasoning about performance r