Researchers have built ProQuant, the first framework that trains multimodal large language models (MLLMs) at very low precision while still keeping vision and language abilities close to full precision. The core idea is to match the fine‑tuning data distribution to the model’s own distribution through a Self‑Relabeling (SRL) step, then gradually turn on quantization starting with vision‐sensitive layers. A vision‑sensitivity vector τ automatically identifies which layers harm image understanding most and activates them first. This Progressive Block Activation (PBA) keeps early text layers in full precision until the model has recovered its visual grounding.

In a direct comparison on Qwen‑VL‑2B‑Instruct, ProQuant achieves an average accuracy of 1,830.28 using 4‑bit weights and 8‑bit activations (W4A8). This outperforms smooth quantization (1,736.61), MBQ (1,810.68), MQuant (1,736.61) and LLM‑QAT (1,931.70). With the even more aggressive W4A4 setting, ProQuant reaches 1,611.94, far higher than MBQ’s 55.41 or LLM‑QAT’s 1,532.13. The post‑training methods MBQ and MQuant collapse under W4A4, scoring near zero on several benchmarks.