---
title: "Discussion on \"Tokens & Tokenizers: How a Computer \"Reads\" a Sentence \" | Hashnode"
description: "Discussion on \"Tokens & Tokenizers: How a Computer \"Reads\" a Sentence \". Here's a weird fact: when you read the sentence \"I love pizza\", your brain doesn't see letters one by one — I, then space, then l, o, v, e... You just see the words. Your brain has already chopped the"
url: "https://hashnode.com/posts/tokens-tokenizers-how-a-computer-reads-a-sentence/6aad91396772d29a6b9e77ca"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Tokens & Tokenizers: How a Computer "Reads" a Sentence](https://sunneysood.hashnode.dev/tokens-tokenizers-how-a-computer-reads-a-sentence?utm_source=hashnode&utm_medium=feed)

Here's a weird fact: when you read the sentence "I love pizza", your brain doesn't see letters one by one — I, then space, then l, o, v, e... You just see the words. Your brain has already chopped the

Sep 18 · 17 min read · sunneysood.hashnode.dev  
Read the article

-   [#token](https://hashnode.com/tag/token)
-   [#tokenization](https://hashnode.com/tag/tokenization)
-   [#llms](https://hashnode.com/tag/llms)
-   [#large-language-models](https://hashnode.com/tag/large-language-models)
-   [#generative-ai](https://hashnode.com/tag/generative-ai)
-   [#python](https://hashnode.com/tag/python)

## Written by

[Sunney Sood](https://hashnode.com/@sunnsood)

@sunnsood · Delhi India

  
[View profile](https://hashnode.com/@sunnsood)

Program Leadership • AI • Cloud • Data Science

Sunney Sood is a Program Manager with experience delivering complex programs in Banking and Insurance. His interests span Program Leadership, Governance, Artificial Intelligence, Data Science, Cloud Technologies, and Digital Transformation. Through hands-on experimentation, industry research, and practical experience, he shares insights that help bridge strategy, technology, and business outcomes. Skills Program Management • Leadership • Governance • Stakeholder Management • Risk Management • Business Transformation • Artificial Intelligence • Data Science • Cloud Technologies • Agile Delivery

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
