---
title: "Discussion on \"What Actually Happens Inside a Transformer Forward Pass\" | Hashnode"
description: "Discussion on \"What Actually Happens Inside a Transformer Forward Pass\". I built an LLM inference engine from scratch in C++."
url: "https://hashnode.com/posts/what-actually-happens-inside-a-transformer-forward-pass/6a9590f681eb2dde2e530822"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [What Actually Happens Inside a Transformer Forward Pass](https://amankarki.hashnode.dev/what-actually-happens-inside-a-transformer-forward-pass?utm_source=hashnode&utm_medium=feed)

I built an LLM inference engine from scratch in C++.

Aug 31 · 10 min read · amankarki.hashnode.dev  
Read the article

-   [#cpp](https://hashnode.com/tag/cpp)
-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#llm](https://hashnode.com/tag/llm)
-   [#cuda](https://hashnode.com/tag/cuda)
-   [#systemprogramming](https://hashnode.com/tag/systemprogramming)

## Written by

[Aman Karki](https://hashnode.com/@amankarki)

@amankarki · India

  
[View profile](https://hashnode.com/@amankarki)

C++/CUDA Engineer · LLM Inference & Systems

C++/CUDA engineer building LLM inference and GPU code from scratch: verbum.cpp, Lattice, RAAG. 2 CUDA PRs merged into llama.cpp. I write about inference, GPU performance, and measuring things honestly.

## Responses

Markdown supported · ⌘↵ to post

No responses yet.