GPTQ
A post-training quantization method for efficient GPU inference.
A post-training quantization method for efficient GPU inference. (M08, M14)
A careful, calibration-based weight-only 4-bit quantization method. Well supported by vLLM.
A post-training quantization method for efficient GPU inference.
A post-training quantization method for efficient GPU inference. (M08, M14)
A careful, calibration-based weight-only 4-bit quantization method. Well supported by vLLM.