---
title: "Discussion on \"Why Tokenization Is Harder for the Myanmar Language\" | Hashnode"
description: "Discussion on \"Why Tokenization Is Harder for the Myanmar Language\". To put it simply, every language model reads text through a chopping step that was mostly trained on English — and Myanmar text walks into that chopper with no spaces, stacked letters, and extra marks"
url: "https://hashnode.com/posts/why-tokenization-is-harder-for-the-myanmar-language/6aeee14eb36743306a426c45"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Why Tokenization Is Harder for the Myanmar Language](https://itsmeminsithu.hashnode.dev/why-tokenization-is-harder-for-the-myanmar-language?utm_source=hashnode&utm_medium=feed)

To put it simply, every language model reads text through a chopping step that was mostly trained on English — and Myanmar text walks into that chopper with no spaces, stacked letters, and extra marks

Nov 7 · 9 min read · itsmeminsithu.hashnode.dev  
Updated Nov 8  
Read the article

-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#artificial-intelligence](https://hashnode.com/tag/artificial-intelligence)
-   [#nlp](https://hashnode.com/tag/nlp)
-   [#language-models](https://hashnode.com/tag/language-models)

## Written by

[Min SiThu](https://hashnode.com/@itsmeminsithu)

@itsmeminsithu · Singapore

  
[View profile](https://hashnode.com/@itsmeminsithu)

Software Engineer

Nubbad Developer. | Now I accidentally create problems.

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
