---
title: "Discussion on \"INT8 Quantization the Second Time Around\" | Hashnode"
description: "Discussion on \"INT8 Quantization the Second Time Around\". I quantized my inference engine's weights to int8 — and almost led with the wrong number before the process itself caught a real bug."
url: "https://hashnode.com/posts/int8-quantization-the-second-time-around/6a99137f61f4f3a3e1dce6ba"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [INT8 Quantization the Second Time Around](https://amankarki.hashnode.dev/int8-quantization-the-second-time-around?utm_source=hashnode&utm_medium=feed)

I quantized my inference engine's weights to int8 — and almost led with the wrong number before the process itself caught a real bug.

Sep 3 · 6 min read · amankarki.hashnode.dev  
Read the article

-   [#cpp](https://hashnode.com/tag/cpp)
-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#llm](https://hashnode.com/tag/llm)
-   [#quantization](https://hashnode.com/tag/quantization)
-   [#systemprogramming](https://hashnode.com/tag/systemprogramming)

## Written by

[Aman Karki](https://hashnode.com/@amankarki)

@amankarki · India

  
[View profile](https://hashnode.com/@amankarki)

C++/CUDA Engineer · LLM Inference & Systems

C++/CUDA engineer building LLM inference and GPU code from scratch: verbum.cpp, Lattice, RAAG. 2 CUDA PRs merged into llama.cpp. I write about inference, GPU performance, and measuring things honestly.

## Responses

Markdown supported · ⌘↵ to post

No responses yet.