---
title: "Discussion on \"Understanding Token and Positional Embeddings in Transformers\" | Hashnode"
description: "Discussion on \"Understanding Token and Positional Embeddings in Transformers\". Transformers, the backbone of many state-of-the-art NLP models such as BERT, GPT has revolutionized the way we approach natural language understanding tasks. One key innovation in transformers is their ability to handle entire sequences of tokens sim..."
url: "https://hashnode.com/posts/understanding-token-and-positional-embeddings-in-transformers/673b732a6fb46456c5c00fa8"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Understanding Token and Positional Embeddings in Transformers](https://rahullokurte.com/understanding-token-and-positional-embeddings-in-transformers?utm_source=hashnode&utm_medium=feed)

Transformers, the backbone of many state-of-the-art NLP models such as BERT, GPT has revolutionized the way we approach natural language understanding tasks. One key innovation in transformers is their ability to handle entire sequences of tokens sim...

Nov 18, 2024 · 4 min read · rahullokurte.com  
Read the article

-   [#nlp](https://hashnode.com/tag/nlp)
-   [#nlp-transformers](https://hashnode.com/tag/nlp-transformers)
-   [#transformers](https://hashnode.com/tag/transformers)

## Written by

[Rahul Lokurte](https://hashnode.com/@rahullokurte)

@rahullokurte · Bengaluru, India

  
[View profile](https://hashnode.com/@rahullokurte)

Cloud and Serverless Enthusiast

I am a Lead Engineer from India. Love to blog about serverless and help teams design and develop serverless architecture. An AWS cloud practitioner. Blogs about AWS Services utilising AWS CDK, CloudFormation, Terraform.

## Responses 1

Markdown supported · ⌘↵ to post

[john ridge](https://hashnode.com/@jcridge1964)

[May 23, 2025](https://hashnode.com/posts/understanding-token-and-positional-embeddings-in-transformers/673b732a6fb46456c5c00fa8/comment/6830897e91e9971aebaa2fee)

## The section that represents the tokens for King and Queen is a bit unclear:

The word "king" might be represented as \[0.5,0.2,0.8,...\]\[0.5, 0.2, 0.8, ...\]\[0.5,0.2,0.8,...\]. The word "queen" might be represented as \[0.6,0.3,0.7,...\]\[0.6, 0.3, 0.7, ...\]\[0.6,0.3,0.7,...\].

## Their closeness in the vector space reflects their semantic similarity.

All of the triples in King are the same (for some reason not clear). Similarly, the triples in Queen are the same.

However, the triples in King could be considered nothing like the triples in Queen.

Could the illustrative numbers in this section be updated please or just slightly differently explained ?