---
title: "#attention articles | Hashnode"
description: "Explore 23 articles tagged #attention on Hashnode. Tutorials, guides, and how-tos from the community."
url: "https://hashnode.com/tag/attention"
---

Tag

# #attention

23 posts · 0 followers

[Rahul Sai Indeevar V](https://hashnode.com/@rahul-ai) · [Sep 7](https://hashnode.com/posts/advanced-transformers-positional-encodings-rope-efficient-attention-mechanisms-kv-caching-mqa-gqa-mla/6a9f04585a20161fc07430bd)

### [Advanced Transformers: Positional Encodings (RoPE) & Efficient Attention Mechanisms (KV Caching, MQA, GQA, MLA)](https://rahul-ai.hashnode.dev/advanced-transformers-positional-encodings-rope-efficient-attention-mechanisms-kv-caching-mqa-gqa-mla?utm_source=hashnode&utm_medium=feed)

In modern Large Language Model (LLM) architectures, two foundational engineering bottlenecks dictate model scalability and inference latency: Representing Order: How self-attention mechanisms—which a

15 min read · [rahul-ai.hashnode.dev](https://rahul-ai.hashnode.dev/advanced-transformers-positional-encodings-rope-efficient-attention-mechanisms-kv-caching-mqa-gqa-mla?utm_source=hashnode&utm_medium=feed)

[3](https://hashnode.com/posts/advanced-transformers-positional-encodings-rope-efficient-attention-mechanisms-kv-caching-mqa-gqa-mla/6a9f04585a20161fc07430bd)

[Rahul Sai Indeevar V](https://hashnode.com/@rahul-ai) · [Sep 2](https://hashnode.com/posts/classical-nlp-to-attention-guide-to-sequence-modeling/6a981b9c6ad679715f674f7d)

### [Classical NLP to Attention: Guide to Sequence Modeling](https://rahul-ai.hashnode.dev/classical-nlp-to-attention-guide-to-sequence-modeling?utm_source=hashnode&utm_medium=feed)

Natural Language Processing (NLP) has undergone a massive paradigm shift over the past decade. We have moved from handcrafted rule-based systems and statistical frequencies to complex sequential archi

11 min read · [rahul-ai.hashnode.dev](https://rahul-ai.hashnode.dev/classical-nlp-to-attention-guide-to-sequence-modeling?utm_source=hashnode&utm_medium=feed)

[Nguyễn Hữu Đức](https://hashnode.com/@ducnguyen-creator) · [Jul 11](https://hashnode.com/posts/draco-ai/6a51b42422570cdfac8f9efd)

### [Building an LLM System from Scratch in Pure Python & NumPy: Architecture, Invariants, and Clean Code, Open-Source 100% Inference Engine and Thinking engine!](https://draco-ai.hashnode.dev/draco-ai?utm_source=hashnode&utm_medium=feed)

Exploring the design invariants, ternary quantization, and self-healing cognitive pipeline of Draco AI, a hardware-agnostic LLM built from scratch.

3 min read · [draco-ai.hashnode.dev](https://draco-ai.hashnode.dev/draco-ai?utm_source=hashnode&utm_medium=feed)

[Mohamed Hamed](https://hashnode.com/@hamed33) · [Apr 21](https://hashnode.com/posts/part-7-the-transformer-the-architecture-that-accidentally-changed-the-world/69e7e9b7be59fd242c7298f3)

### [Part 7 — The Transformer: The Architecture That Accidentally Changed the World](https://mo-mentor.hashnode.dev/part-7-the-transformer-the-architecture-that-accidentally-changed-the-world?utm_source=hashnode&utm_medium=feed)

THE ENGINE OF THE FUTURE Transformer "Attention Is All You Need" — the paper that changed everything Last article we saw how the four learning types + training loop built ChatGPT. Today we open the box and see the exact architecture that made all of ...

16 min read · [mo-mentor.hashnode.dev](https://mo-mentor.hashnode.dev/part-7-the-transformer-the-architecture-that-accidentally-changed-the-world?utm_source=hashnode&utm_medium=feed)

[Aditya Gupta](https://hashnode.com/@adiyogiarts) · [Mar 28](https://hashnode.com/posts/identifying-early-warning-signs-of-attention-mecha/69c76d224f9429061582f4bf)

### [Identifying Early Warning Signs of Attention Mechanism Instability](https://adiyogiarts.hashnode.dev/identifying-early-warning-signs-of-attention-mecha?utm_source=hashnode&utm_medium=feed)

Discover how to spot and resolve attention mechanism instability in AI models.

6 min read · [adiyogiarts.hashnode.dev](https://adiyogiarts.hashnode.dev/identifying-early-warning-signs-of-attention-mecha?utm_source=hashnode&utm_medium=feed)

[Aditya Gupta](https://hashnode.com/@adiyogiarts) · [Mar 28](https://hashnode.com/posts/the-intricate-dance-of-self-attention-what-can-go/69c76d1b6f3982e4a4a69446)

### [The Intricate Dance of Self-Attention: What Can Go Wrong?](https://adiyogiarts.hashnode.dev/the-intricate-dance-of-self-attention-what-can-go?utm_source=hashnode&utm_medium=feed)

into Transformer model failures and how attention mechanisms break down.

6 min read · [adiyogiarts.hashnode.dev](https://adiyogiarts.hashnode.dev/the-intricate-dance-of-self-attention-what-can-go?utm_source=hashnode&utm_medium=feed)

[Aditya Gupta](https://hashnode.com/@adiyogiarts) · [Mar 28](https://hashnode.com/posts/the-bottleneck-of-dense-attention-in-long-contexts/69c76d1434bfea852852443c)

### [The Bottleneck of Dense Attention in Long Contexts](https://adiyogiarts.hashnode.dev/the-bottleneck-of-dense-attention-in-long-contexts?utm_source=hashnode&utm_medium=feed)

Discover DeepSeek Sparse Attention, a technique allowing LLMs to handle 1M+ tokens and halve costs.

7 min read · [adiyogiarts.hashnode.dev](https://adiyogiarts.hashnode.dev/the-bottleneck-of-dense-attention-in-long-contexts?utm_source=hashnode&utm_medium=feed)

[Kashif](https://hashnode.com/@ifkash) · [Feb 8](https://hashnode.com/posts/smol-llama-pretraining/698898652fb502a9301f0a63)

### [I Pretrained a 360M LLaMA-Style Language Model from Scratch on 6B FineWeb Tokens (Single H100)](https://blog.ifkash.dev/smol-llama-pretraining?utm_source=hashnode&utm_medium=feed)

Explore pretraining a 360M LLaMA-style language model on 6B tokens using a single H100 GPU, with reproducible results

5 min read · [blog.ifkash.dev](https://blog.ifkash.dev/smol-llama-pretraining?utm_source=hashnode&utm_medium=feed)

[cicada](https://hashnode.com/@cicada) · [Dec 31, 2025](https://hashnode.com/posts/is-attention-obsolete/69551bd8f7c5039c93227172)

### [Is Attention Obsolete?](https://chinmayblog.cnresearchs.in/is-attention-obsolete?utm_source=hashnode&utm_medium=feed)

Meet Grassmann Flows: A Geometric, Attention‑Free Path Beyond Transformers

6 min read · [chinmayblog.cnresearchs.in](https://chinmayblog.cnresearchs.in/is-attention-obsolete?utm_source=hashnode&utm_medium=feed)

[Gonzalo](https://hashnode.com/@gonzalosanchezpla) · [Oct 7, 2025](https://hashnode.com/posts/how-large-language-models-changed-ai-forever/68e4b5dfb844a07778bc18f7)

### [How Large Language Models Changed AI Forever](https://thinkingmachines.hashnode.dev/how-large-language-models-changed-ai-forever?utm_source=hashnode&utm_medium=feed)

Discover how Large Language Models work: from embeddings and attention to prediction. The architecture that transformed modern AI.

8 min read · [thinkingmachines.hashnode.dev](https://thinkingmachines.hashnode.dev/how-large-language-models-changed-ai-forever?utm_source=hashnode&utm_medium=feed)
