---
title: "Discussion on \"Grokking of Transformer on Modular Addition\" | Hashnode"
description: "Discussion on \"Grokking of Transformer on Modular Addition\". Background\nGrokking is a phenomenon where a model quickly achieves near-perfect training accuracy (memorization), while validation accuracy remains near chance for a long time, and then later transitions sharply to strong generalization after extende..."
url: "https://hashnode.com/posts/grokking-of-transformer-on-modular-addition/69891e062bb00818234f0baa"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Grokking of Transformer on Modular Addition](https://taot.hashnode.dev/grokking-of-transformer-on-modular-addition?utm_source=hashnode&utm_medium=feed)

Background Grokking is a phenomenon where a model quickly achieves near-perfect training accuracy (memorization), while validation accuracy remains near chance for a long time, and then later transitions sharply to strong generalization after extende...

Feb 8 · 3 min read · taot.hashnode.dev  
Read the article

-   [#grokking](https://hashnode.com/tag/grokking)
-   [#transformer](https://hashnode.com/tag/transformer)

## Written by

[Terry Tao](https://hashnode.com/@taot)

@taot

  
[View profile](https://hashnode.com/@taot)

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
