Multiverse Computing introduced Quantization-Aware Healing, a technique that produces 4-bit compressed large language models that outperform their full-precision originals, according to a report published on Hugging Face. The method applies knowledge distillation directly from the original uncompressed model to recover capabilities typically lost during compression.
Standard deployment pipelines compress model architectures by pruning layers, heads, or neurons, and then quantize weights to 4-bit formats before running recovery training. Existing quantization-aware distillation methods train the quantized model against the compressed bfloat16 checkpoint. Multiverse Computing instead distills directly from the original full-size, full-precision teacher into the half-size student running in MXFP4, using a Kullback-Leibler divergence loss on output logits.
Benchmark results
Researchers tested the approach on a GPT-OSS 120B model compressed to 60 billion parameters. The resulting 4-bit MXFP4 model beat its recovered 60-billion-parameter bfloat16 baseline on 7 of 9 evaluated benchmarks. On the AA-LCR long-context reasoning benchmark, the 4-bit model scored 42.7 compared to 35.3 for the bfloat16 version. On AIME 2025 mathematics problems, it scored 76.3 compared to 70.7. The 4-bit model trailed the 16-bit version on only two benchmarks, MMLU-Pro and SciCode, by less than 1.5 points. It also surpassed the full-size 120-billion-parameter teacher on LiveCodeBench, scoring 66.5 against 66.0.
Training speed and stability
Multiverse Computing also evaluated the approach against standard quantization-aware training on a GPT-OSS 9B model quantized to MXFP4. Both methods achieved similar peak performance across MMLU-Pro, LiveCodeBench, and GPQA Diamond, with the healing technique reaching 54.9 and standard training reaching 54.6.
The two methods diverged substantially in training efficiency and stability. The healing recipe reached its peak in roughly 100 steps and stayed within two points of that level for the remainder of training. Standard quantization-aware training required roughly 700 steps to reach its peak before declining by nearly 19 points by step 1,200.
To process long sequences during healing, the pipeline incorporates a memory-efficient chunked loss capable of handling documents up to 32,000 tokens without materializing the full vocabulary grid. That mechanism draws on Multiverse Computing's companion research on scalable distillation published on August 10, 2026.
