---
title: "Discussion on \"Counting tokens at scale using tiktoken\" | Hashnode"
description: "Discussion on \"Counting tokens at scale using tiktoken\". Using encode_to_numpy to count number of tokens in large strings to save memory and CPU"
url: "https://hashnode.com/posts/counting-tokens-at-scale-using-tiktoken/687fbe949e43654faf65ec8a"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Counting tokens at scale using tiktoken](https://www.dsdev.in/counting-tokens-at-scale-using-tiktoken?utm_source=hashnode&utm_medium=feed)

Using encode\_to\_numpy to count number of tokens in large strings to save memory and CPU

Jul 22, 2025 · 2 min read · dsdev.in  
Read the article

-   [#tokenization](https://hashnode.com/tag/tokenization)
-   [#llms](https://hashnode.com/tag/llms)
-   [#tiktoken](https://hashnode.com/tag/tiktoken)

## Written by

[Dhaval Singh](https://hashnode.com/@DhavalSingh)

@DhavalSingh · Bangalore, India

  
[View profile](https://hashnode.com/@DhavalSingh)

Writing about Software Development, Backend in particular!

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
