---
title: "Discussion on \"How Query, Key, and Value (QKV) Work in Transformers\" | Hashnode"
description: "Discussion on \"How Query, Key, and Value (QKV) Work in Transformers\". Every Transformer-based language model, from BERT to GPT, relies on a mechanism called attention to determine how different tokens in a sequence relate to one another. At the heart of this mechanism a"
url: "https://hashnode.com/posts/how-query-key-and-value-qkv-work-in-transformers/6ad9c78da2619bfccea4aca7"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [How Query, Key, and Value (QKV) Work in Transformers](https://nishanthan.hashnode.dev/how-query-key-and-value-qkv-work-in-transformers?utm_source=hashnode&utm_medium=feed)

Every Transformer-based language model, from BERT to GPT, relies on a mechanism called attention to determine how different tokens in a sequence relate to one another. At the heart of this mechanism a

Oct 22 · 7 min read · nishanthan.hashnode.dev  
Read the article

-   [#ai](https://hashnode.com/tag/ai)
-   [#deeplearning](https://hashnode.com/tag/deeplearning)
-   [#transformers](https://hashnode.com/tag/transformers)
-   [#nlp](https://hashnode.com/tag/nlp)
-   [#llm](https://hashnode.com/tag/llm)

## Written by

[Nishanthan Karunakaran](https://hashnode.com/@nishanthan)

@nishanthan

  
[View profile](https://hashnode.com/@nishanthan)

An Software Engineer who loves to build end to end working products

## Responses

Markdown supported · ⌘↵ to post

No responses yet.