---
title: "Discussion on \"How Kimi K3 reads a million tokens\" | Hashnode"
description: "Discussion on \"How Kimi K3 reads a million tokens\". The road from softmax to Kimi Delta Attention — and why deleting positional encoding was worth 5.5 points."
url: "https://hashnode.com/posts/how-kimi-k3-reads-a-million-tokens/6a769f4eefa957e947735e73"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [How Kimi K3 reads a million tokens](https://minhleduc.hashnode.dev/how-kimi-k3-reads-a-million-tokens?utm_source=hashnode&utm_medium=feed)

The road from softmax to Kimi Delta Attention — and why deleting positional encoding was worth 5.5 points.

Aug 8 · 12 min read · minhleduc.hashnode.dev  
Read the article

## Written by

[minhleduc](https://hashnode.com/@firefrog)

@firefrog · Ho Chi Minh City, Vietnam

  
[View profile](https://hashnode.com/@firefrog)

Just another Dev guy.

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
