---
title: "Discussion on \"Speculative Decoding: From Theory to Implementation\" | Hashnode"
description: "Discussion on \"Speculative Decoding: From Theory to Implementation\". The complete guide to understanding the concept of speculative decoding in LLM inference and implementing it from scratch"
url: "https://hashnode.com/posts/speculative-decoding/690b1ada1921d85b235d9b58"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Speculative Decoding: From Theory to Implementation](https://galacodes.hashnode.dev/speculative-decoding?utm_source=hashnode&utm_medium=feed)

The complete guide to understanding the concept of speculative decoding in LLM inference and implementing it from scratch

Nov 5, 2025 · 15 min read · galacodes.hashnode.dev  
Read the article

-   [#ai](https://hashnode.com/tag/ai)
-   [#llm](https://hashnode.com/tag/llm)
-   [#chatgpt](https://hashnode.com/tag/chatgpt)

## Written by

[Jay Gala](https://hashnode.com/@Jaygala223)

@Jaygala223 · Mumbai, India

  
[View profile](https://hashnode.com/@Jaygala223)

AI Software Engineer at Intel

Currently working as an AI Software Engineer at Intel

## Responses 1

Markdown supported · ⌘↵ to post

[Deekshith Reddy](https://hashnode.com/@birdman)

[Nov 10, 2025](https://hashnode.com/posts/speculative-decoding/690b1ada1921d85b235d9b58/comment/6911704248e22e4d40ea1455)

Great article! Found a small bug, adjusted\_probs = torch.clamp( target\_probs - torch.softmax(all\_logits\[position\], dim=0), min=0.0 ) it should be target\_probs - draft\_probs You're essentially doing target\_probs - target\_probs, which means we're sampling from your else case, which means we're sampling from the target\_probs but also in the rejected overlap space with draft model. If we do it correctly, we will be sampling from p'(t) which will be in the target\_space not overlapping with draft model's space.
