---
title: "Discussion on \"From 8 Bits to 4: Sidecar, MoE, and the imatrix Trick That Worked\" | Hashnode"
description: "Discussion on \"From 8 Bits to 4: Sidecar, MoE, and the imatrix Trick That Worked\". Pushing SCLP exponent-palette compression to 4 bits/weight: per-block palettes, a fused MoE GEMV, and an imatrix sidecar that beats Q4_K."
url: "https://hashnode.com/posts/from-8-bits-to-4-sidecar-moe-and-the-imatrix-trick-that-worked/6a20ba490b1b181bac25f4b1"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [From 8 Bits to 4: Sidecar, MoE, and the imatrix Trick That Worked](https://blog.kerchum.dev/from-8-bits-to-4-sidecar-moe-and-the-imatrix-trick-that-worked?utm_source=hashnode&utm_medium=feed)

Pushing SCLP exponent-palette compression to 4 bits/weight: per-block palettes, a fused MoE GEMV, and an imatrix sidecar that beats Q4\_K.

Jun 3 · 9 min read · blog.kerchum.dev  
Read the article

-   [#llm](https://hashnode.com/tag/llm)
-   [#deep-learning](https://hashnode.com/tag/deep-learning)
-   [#quantization](https://hashnode.com/tag/quantization)
-   [#opensource](https://hashnode.com/tag/opensource)

## Written by

[Alexander Kerchum](https://hashnode.com/@ajkerchum)

@ajkerchum

  
[View profile](https://hashnode.com/@ajkerchum)

Senior Software Engineer at Upstart

## Responses

Markdown supported · ⌘↵ to post

No responses yet.
