---
title: "Discussion on \"Tokenization Basics: What the Model Actually Reads\" | Hashnode"
description: "Discussion on \"Tokenization Basics: What the Model Actually Reads\". Building a Transformer from Scratch, Part 1"
url: "https://hashnode.com/posts/transformer-from-scratch-part-1-tokenization/6af4682fa511956f5e6a7d34"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Tokenization Basics: What the Model Actually Reads](https://yashksingh.hashnode.dev/transformer-from-scratch-part-1-tokenization?utm_source=hashnode&utm_medium=feed)

Building a Transformer from Scratch, Part 1

Nov 11 · 8 min read · yashksingh.hashnode.dev  
Updated 19h ago  
Read the article

-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#nlp](https://hashnode.com/tag/nlp)
-   [#python](https://hashnode.com/tag/python)
-   [#ai](https://hashnode.com/tag/ai)
-   [#llm](https://hashnode.com/tag/llm)

## Written by

[Yash Singh](https://hashnode.com/@yashks)

@yashks · Mumbai, India

  
[View profile](https://hashnode.com/@yashks)

Final-year CSE (AI/ML) student · building language models from scratch

I’m Yash, a final-year Computer Science student specializing in AI and ML. I like understanding how things work underneath, so I built a GPT-style transformer from scratch, tokenizer included, and trained it at 110M and 336M parameters. I write about what I learn along the way. Interested in NLP, LLMs, interpretability and healthcare AI.

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
