---
title: "Discussion on \"Building BPE from Scratch: How a Tokenizer Learns Its Vocabulary\" | Hashnode"
description: "Discussion on \"Building BPE from Scratch: How a Tokenizer Learns Its Vocabulary\". Building a Transformer from Scratch, Part 2"
url: "https://hashnode.com/posts/transformer-from-scratch-part-2-bpe/6afec44f0c6c32beffae2174"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Building BPE from Scratch: How a Tokenizer Learns Its Vocabulary](https://yashksingh.hashnode.dev/transformer-from-scratch-part-2-bpe?utm_source=hashnode&utm_medium=feed)

Building a Transformer from Scratch, Part 2

19h ago · 23 min read · yashksingh.hashnode.dev  
Read the article

-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#nlp](https://hashnode.com/tag/nlp)
-   [#python](https://hashnode.com/tag/python)
-   [#ai](https://hashnode.com/tag/ai)
-   [#ml](https://hashnode.com/tag/ml)

## Written by

[Yash Singh](https://hashnode.com/@yashks)

@yashks · Mumbai, India

  
[View profile](https://hashnode.com/@yashks)

Final-year CSE (AI/ML) student · building language models from scratch

I’m Yash, a final-year Computer Science student specializing in AI and ML. I like understanding how things work underneath, so I built a GPT-style transformer from scratch, tokenizer included, and trained it at 110M and 336M parameters. I write about what I learn along the way. Interested in NLP, LLMs, interpretability and healthcare AI.

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
