---
title: "Discussion on \"Tucker Attention: GQA, MLA, and MHA Were the Same Thing All Along\" | Hashnode"
description: "Discussion on \"Tucker Attention: GQA, MLA, and MHA Were the Same Thing All Along\". All major attention variants are special cases of one tensor decomposition, achieving 10x parameter reduction with zero performance loss"
url: "https://hashnode.com/posts/tucker-attention-gqa-mla-mha-same-thing-all-along/69cec5402d02ee22cb0df18f"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Tucker Attention: GQA, MLA, and MHA Were the Same Thing All Along](https://theagentstack.hashnode.dev/tucker-attention-gqa-mla-mha-same-thing-all-along?utm_source=hashnode&utm_medium=feed)

All major attention variants are special cases of one tensor decomposition, achieving 10x parameter reduction with zero performance loss

Apr 2 · 7 min read · theagentstack.hashnode.dev  
Read the article

-   [#artificial-intelligence](https://hashnode.com/tag/artificial-intelligence)
-   [#deep-learning](https://hashnode.com/tag/deep-learning)
-   [#llm](https://hashnode.com/tag/llm)
-   [#machine-learning](https://hashnode.com/tag/machine-learning)

## Written by

[Manas Vardhan](https://hashnode.com/@manasvardhan)

@manasvardhan · Los Angeles, USA

  
[View profile](https://hashnode.com/@manasvardhan)

I write philosophy, literature and tech

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
