Researchers found that the entropy of a low‑precision decoder’s output distribution reliably indicates when a token chosen with a 4‑bit approximation differs from the 8‑bit reference. In a study on the Qwen3‑0.6B model running the GSM8K benchmark, the entropy gate achieved an area under the ROC curve of 0.928 (95 % interval 0.924‑0.932) for detecting such argmax flips.

The method stores all weights in 8 bits but first decodes only the leading 4 bits. A low‑pass step is performed for every token, and the decoder escalates to a full 8‑bit pass only when the low‑precision distribution shows high entropy or the top two logits are close. On 36,724 replayed steps, this approach predicts flips with an AUC of 0.93.

When the entropy gate is allowed to trigger escalations at a 15 % budget, it captures 78 % of flips and 59 % of Kullback‑Leibler mass, compared with 94 % of flips and 55 % of KL for a margin‑based gate, and 15 % for random escalation. These numbers come from live decoding of 150 GSM8K problems.

Limitations exist. The work was performed only on a single small model (Qwen3‑0.6B); larger models, other families, and mixture‑of‑expert architectures were not evaluated. The benchmark involved greedy decoding capped at 384 tokens, which may influence error patterns.

Future studies should test the entropy gate on bigger models and diverse decoding settings and measure wall‑clock cost, not just weight‑byte usage.