---
title: "Discussion on \"Tokens & Tokenization: Why AI \"Counts\" Words Weirdly\" | Hashnode"
description: "Discussion on \"Tokens & Tokenization: Why AI \"Counts\" Words Weirdly\". Tokens & Tokenization: Why AI Counts Words Weirdly"
url: "https://hashnode.com/posts/tokens-tokenization-why-ai-counts-words-weirdly/6ab4cf96ce67b7fbc2815d87"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Tokens & Tokenization: Why AI "Counts" Words Weirdly](https://nagspidey.hashnode.dev/tokens-tokenization-why-ai-counts-words-weirdly?utm_source=hashnode&utm_medium=feed)

Tokens & Tokenization: Why AI Counts Words Weirdly

Sep 24 · 8 min read · nagspidey.hashnode.dev  
Read the article

-   [#ai](https://hashnode.com/tag/ai)
-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#llm](https://hashnode.com/tag/llm)
-   [#tokenization](https://hashnode.com/tag/tokenization)
-   [#nlp](https://hashnode.com/tag/nlp)

## Written by

[Nagappan S](https://hashnode.com/@imkarthik03)

@imkarthik03 · Chennai, India

  
[View profile](https://hashnode.com/@imkarthik03)

Building scalable products at the intersection of Full Stack Engineering & AI

I build scalable web apps, experiment with AI, and turn complex engineering problems into working products.

## Responses 1

Markdown supported · ⌘↵ to post

[Brian · AI News](https://hashnode.com/@brianainews)

[Sep 24](https://hashnode.com/posts/tokens-tokenization-why-ai-counts-words-weirdly/6ab4cf96ce67b7fbc2815d87/comment/6ab53a16f87500c6102b5e96)

Token burn is the real AI coding tax. I log cost per merged PR so a flashy agent that wastes retries shows up in the budget before the demo does.
