Quantization: GPTQ vs AWQ

Quantization reduces the memory footprint of an LLM by converting weight numerical values from 16-bit floating points (FP16) to 4-bit or 8-bit integers. This allows massive models to run on affordable