Lessons From the Bottom of the Stack: Shipping a QuantThe SCLP compression algorithm — palette the exponents, sidecar the outliers, pack the rest — was a week or so of prototyping. The two posts before this one covered it end to end. This post is about tJun 5 · 12 min readblog.kerchum.dev
From 8 Bits to 4: Sidecar, MoE, and the imatrix Trick That WorkedLast time we cut BF16 weights in half by treating the exponent as a 16-entry palette instead of an 8-bit field. SCLP8: 7.9 GB instead of 15.0, perplexity slightly better than the original, token generJun 3 · 9 min readblog.kerchum.dev
LLMs Use Just 16 of 256 Exponents — So We Compressed the Rest AwayMost people compressing LLM weights are fighting the same war: squeeze 7 billion floats into less memory without wrecking the model. The standard weapons are quantization schemes — map each float to a2May 29 · 9 min readblog.kerchum.dev
How to move a directory from one git repo to another (or new) without losing historyMake copy of repo git clone dirtySourceRepo newSourceRepo OR clone from actual git repo and prevent push git remote set-url --push origin no_push Make sure to checkout the correct branch before the next step. Cloning from another local directory al...Nov 24, 2021 · 1 min readblog.kerchum.dev
