---
title: "Discussion on \"Tokenization Simplified: Part-2\" | Hashnode"
description: "Discussion on \"Tokenization Simplified: Part-2\". nlp tokenization, tokenization guide, tokens in nlp"
url: "https://hashnode.com/posts/tokenization-simplified-part-2/66c3574d71e87702d4e5b6bd"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Tokenization Simplified: Part-2](https://j-jayasri.hashnode.dev/tokenization-simplified-part-2?utm_source=hashnode&utm_medium=feed)

nlp tokenization, tokenization guide, tokens in nlp

Aug 19, 2024 · 8 min read · j-jayasri.hashnode.dev  
Read the article

-   [#large-language-models](https://hashnode.com/tag/large-language-models)
-   [#generative-ai](https://hashnode.com/tag/generative-ai)
-   [#tokenization](https://hashnode.com/tag/tokenization)
-   [#tokens](https://hashnode.com/tag/tokens)
-   [#nlp](https://hashnode.com/tag/nlp)
-   [#natural-language-processing](https://hashnode.com/tag/natural-language-processing)
-   [#ai](https://hashnode.com/tag/ai)

## Written by

[Jayasri](https://hashnode.com/@JayasriJ)

@JayasriJ

  
[View profile](https://hashnode.com/@JayasriJ)

ML Engineer (GenAI)

I'm currently working as ML Engineer (GenAI) with 2.5+ years experience in building End to End AI Systems for startups

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
