--- title: "#attention-mechanism articles | Hashnode" description: "Explore 98 articles tagged #attention-mechanism on Hashnode. Tutorials, guides, and how-tos from the community." url: "https://hashnode.com/tag/attention-mechanism" --- Tag # #attention-mechanism 98 posts · 1 followers [Rounak Show](https://hashnode.com/@ShowRounak) · [2d ago](https://hashnode.com/posts/why-do-we-apply-softmax-to-qk-before-multiplying-by-v/6af1b85942416b89d2ff0a96) ### [Inside Attention: Why Softmax Sits Between QKᵀ and V](https://rounakshow.hashnode.dev/why-do-we-apply-softmax-to-qk-before-multiplying-by-v?utm_source=hashnode&utm_medium=feed) When I first learned self-attention, the formula looked straightforward: $$\\operatorname{Attention}(Q,K,V)=\\operatorname{softmax}\\left(\\frac{QK^\\top}{\\sqrt{d\_k}}+M\\right)V$$ But one question bothered 6 min read · [rounakshow.hashnode.dev](https://rounakshow.hashnode.dev/why-do-we-apply-softmax-to-qk-before-multiplying-by-v?utm_source=hashnode&utm_medium=feed) [Nishanthan Karunakaran](https://hashnode.com/@nishanthan) · [Oct 15](https://hashnode.com/posts/understanding-transformer-the-architecture-behind-modern-ai/6ad04aba652296781d0156e8) ### [Understanding Transformer: The Architecture Behind Modern AI](https://nishanthan.hashnode.dev/understanding-transformer-the-architecture-behind-modern-ai?utm_source=hashnode&utm_medium=feed) What Is a Transformer? A Transformer is a neural network architecture introduced by Google researchers in 2017 in the paper Attention Is All You Need. It uses attention mechanisms to help the model le 6 min read · [nishanthan.hashnode.dev](https://nishanthan.hashnode.dev/understanding-transformer-the-architecture-behind-modern-ai?utm_source=hashnode&utm_medium=feed) [Ekansh Nangia](https://hashnode.com/@ekanshn) · [Sep 17](https://hashnode.com/posts/introduction-to-mechanistic-interpretability/6aaba1e727577e90301e1109) ### [Introduction To Mechanistic Interpretability](https://ekanshn.hashnode.dev/introduction-to-mechanistic-interpretability?utm_source=hashnode&utm_medium=feed) How do LLMs actually work? Explore mechanistic interpretability, AI epistemology, and why probing neural networks cannot crack the AI black box. 18 min read · [ekanshn.hashnode.dev](https://ekanshn.hashnode.dev/introduction-to-mechanistic-interpretability?utm_source=hashnode&utm_medium=feed) [Sunney Sood](https://hashnode.com/@sunnsood) · [Sep 11](https://hashnode.com/posts/how-large-language-models-actually-work-from-your-prompt-to-every-word-they-write/6aa45643c22712d0ba8b153e) ### [How Large Language Models Actually Work: From Your Prompt to Every Word They Write](https://sunneysood.hashnode.dev/how-large-language-models-actually-work-from-your-prompt-to-every-word-they-write?utm_source=hashnode&utm_medium=feed) A complete, plain-English mental model — from tokens to attention to the exact moment ChatGPT picks its next word. When you type a question into ChatGPT and an answer appears a few seconds later, it 26 min read · [sunneysood.hashnode.dev](https://sunneysood.hashnode.dev/how-large-language-models-actually-work-from-your-prompt-to-every-word-they-write?utm_source=hashnode&utm_medium=feed) [Rahul Sai Indeevar V](https://hashnode.com/@rahul-ai) · [Sep 7](https://hashnode.com/posts/advanced-transformers-positional-encodings-rope-efficient-attention-mechanisms-kv-caching-mqa-gqa-mla/6a9f04585a20161fc07430bd) ### [Advanced Transformers: Positional Encodings (RoPE) & Efficient Attention Mechanisms (KV Caching, MQA, GQA, MLA)](https://rahul-ai.hashnode.dev/advanced-transformers-positional-encodings-rope-efficient-attention-mechanisms-kv-caching-mqa-gqa-mla?utm_source=hashnode&utm_medium=feed) In modern Large Language Model (LLM) architectures, two foundational engineering bottlenecks dictate model scalability and inference latency: Representing Order: How self-attention mechanisms—which a 15 min read · [rahul-ai.hashnode.dev](https://rahul-ai.hashnode.dev/advanced-transformers-positional-encodings-rope-efficient-attention-mechanisms-kv-caching-mqa-gqa-mla?utm_source=hashnode&utm_medium=feed) [3](https://hashnode.com/posts/advanced-transformers-positional-encodings-rope-efficient-attention-mechanisms-kv-caching-mqa-gqa-mla/6a9f04585a20161fc07430bd) [Rahul Sai Indeevar V](https://hashnode.com/@rahul-ai) · [Sep 2](https://hashnode.com/posts/classical-nlp-to-attention-guide-to-sequence-modeling/6a981b9c6ad679715f674f7d) ### [Classical NLP to Attention: Guide to Sequence Modeling](https://rahul-ai.hashnode.dev/classical-nlp-to-attention-guide-to-sequence-modeling?utm_source=hashnode&utm_medium=feed) Natural Language Processing (NLP) has undergone a massive paradigm shift over the past decade. We have moved from handcrafted rule-based systems and statistical frequencies to complex sequential archi 11 min read · [rahul-ai.hashnode.dev](https://rahul-ai.hashnode.dev/classical-nlp-to-attention-guide-to-sequence-modeling?utm_source=hashnode&utm_medium=feed) [Vijay Ram Enaganti](https://hashnode.com/@vijay-ram-enag) · [Aug 31](https://hashnode.com/posts/sequential-attention/6a958669cc91e42ff4ace729) ### [Reproducing Sequential Attention](https://vijay-ram.hashnode.dev/sequential-attention?utm_source=hashnode&utm_medium=feed) (and finding out where my reproduction disagrees with Google's) 9 min read · [vijay-ram.hashnode.dev](https://vijay-ram.hashnode.dev/sequential-attention?utm_source=hashnode&utm_medium=feed) [Viv Chaudhary](https://hashnode.com/@chuadharyviv) · [Aug 22](https://hashnode.com/posts/five-mental-models-that-make-modern-ai-click/6a89e6f47fb7c33bd12d3aa2) ### [Five Mental Models That Make Modern AI Click](https://truenorthbits.hashnode.dev/five-mental-models-that-make-modern-ai-click?utm_source=hashnode&utm_medium=feed) Tokens, context windows, embeddings, attention, and scale - the five ideas that stop new model releases from feeling like starting over 12 min read · [truenorthbits.hashnode.dev](https://truenorthbits.hashnode.dev/five-mental-models-that-make-modern-ai-click?utm_source=hashnode&utm_medium=feed) [Akash](https://hashnode.com/@irsakash) · [Aug 5](https://hashnode.com/posts/abstractive-text-summarization-with-attention-mechanism-seq2seq-model/6a73826adfba9860c7bb9682) ### [Abstractive Text Summarization with Attention Mechanism: Seq2Seq Model for Academic Papers](https://akashbuilds.hashnode.dev/abstractive-text-summarization-with-attention-mechanism-seq2seq-model?utm_source=hashnode&utm_medium=feed) Build an end-to-end abstractive text summarization system using Seq2Seq LSTM encoder-decoder with attention mechanism to generate concise summaries. 7 min read · [akashbuilds.hashnode.dev](https://akashbuilds.hashnode.dev/abstractive-text-summarization-with-attention-mechanism-seq2seq-model?utm_source=hashnode&utm_medium=feed) [Piyush Jadhav](https://hashnode.com/@piyush3191) · [Aug 3](https://hashnode.com/posts/attention-mechanism-in-deep-learning-the-foundation-of-transformers/6a705add5e665c96ac27e98c) ### [Attention Mechanism in Deep Learning: The Foundation of Transformers](https://attentionmechanism.hashnode.dev/attention-mechanism-in-deep-learning-the-foundation-of-transformers?utm_source=hashnode&utm_medium=feed) Learn how Attention Mechanisms enable AI models to focus on the most relevant information, improving accuracy and powering modern applications like ChatGPT, BERT, and machine translation. 4 min read · [attentionmechanism.hashnode.dev](https://attentionmechanism.hashnode.dev/attention-mechanism-in-deep-learning-the-foundation-of-transformers?utm_source=hashnode&utm_medium=feed)