SoScale: Second-Order Scale Search for FP4 Quantization

Pengxiang Zhao, Xing Li, Hui-Ling Zhen, Weizhe Lin, Xianzhi Yu, Zhenhua Dong
23 minread4.20scoreNo. 311-16 Infra issue
Contents · 7 sections1FP4's 16 values get used up by outliers2What looking only at in-block error misses3How SoScale picks a scale4Walking through it: the weight block in Figure 15Results6What this means for people deploying models7Limitations
Rel 5Nov 4Prac 4Evid 3Time 4

You quantize a model to 4 bits and pick the scale with smaller error, so why does the model get worse? SoScale picks it by the layer outputYou are doing FP4 quantization and switch to a finer scale search: for every weight block, pick the scale with the smallest reconstruction error. Block by block, the error really does go down. Then you measure perplexity, and it comes out slightly higher than plain default quantization.

The paper measured the same thing on Qwen3-8B: under MXFP4, the method that picks scales by element-wise error has higher perplexity than default quantization; picking by layer output instead brings perplexity down to about 10.1, much closer to BF16's 9.7.

This paper, from Huawei, is about FP4 post-training quantization of large models. Each FP4 number has only 16 possible values, and formats such as MXFP4, NVFP4 and HiF4 give each small block of numbers a scale that decides where those 16 values sit on the number line. The default rule sets the scale from the largest absolute value in the block; existing scale-search methods (such as ScaleSearch) try a few candidates and pick the one with the smallest mean squared error within the block. SoScale changes the scoring: it weights each error with a Hessian computed from calibration data to see how far it would move the layer output, and adds GPTQ-style error compensation, so later blocks absorb the error of earlier ones. On Qwen3-14B it keeps 97.17%, 99.20% and 99.40% of BF16 average accuracy under the three formats, the best in every group.

A scale should be chosen by its effect on the layer output, and the smallest in-block error does not give the best model; the gains are modest, 0.21 to 0.77 percentage points above the strongest non-SoScale baseline.

FP4's 16 values get used up by outliers

An FP4 number itself has only 4 bits. MXFP4 uses the E2M1 encoding, which has only 8 possible values on the positive side, the largest being 6, multiplied by the scale shared by the block. Because an MXFP4 scale can only be an integer power of 2 and is shared by every 32 numbers, each step of the scale doubles or halves the whole grid.

A large scale covers a wide range but leaves big gaps between neighboring values; a small scale covers a narrow range but resolves values near 0 more finely. To avoid clipping the largest number, the default rule sets the scale from the block's largest absolute value. When a block has one or two outliers, the whole grid is stretched, and most of the numbers crowded near 0 get only a few grid points.

A real MXFP4 weight block from Qwen3-8B. Gray bars are the value distribution: most values crowd near 0, with one outlier around −0.4 on the left. Blue ticks are the possible values under the default scale 0.125; orange ticks are the possible values under the searched scale 0.0625. Source: paper Figure 1
A real MXFP4 weight block from Qwen3-8B. Gray bars are the value distribution: most values crowd near 0, with one outlier around −0.4 on the left. Blue ticks are the possible values under the default scale 0.125; orange ticks are the possible values under the searched scale 0.0625. Source: paper Figure 1

What looking only at in-block error misses

Existing scale search tries a few candidate scales in a small window and picks the one with the smallest in-block mean squared error. It assumes every weight's error matters equally.

But a layer's output is input times weights. If the input channel for one column of weights is often large on real data, that column's error gets amplified; if that input is almost always 0, even a large error barely affects the output. Errors in two columns can also cancel or add up, depending on whether the two input channels tend to appear together. In-block mean squared error sees none of this.

The right panel of Figure 1 in the paper shows the consequence. Under MXFP4, ScaleSearch, which picks scales by mean squared error, has a perplexity of about 11.0, higher than default quantization at about 10.9.

Perplexity of Qwen3-8B under three FP4 formats. The blue line is default quantization, the dashed orange line is ScaleSearch, which picks scales by mean squared error, the green line is SoScale, and the gray dotted line is BF16. Source: paper Figure 1
Perplexity of Qwen3-8B under three FP4 formats. The blue line is default quantization, the dashed orange line is ScaleSearch, which picks scales by mean squared error, the green line is SoScale, and the gray dotted line is BF16. Source: paper Figure 1

How SoScale picks a scale

First, run 128 WikiText calibration sequences of 2,048 tokens each, collect the input X of every linear layer, and compute H = XᵀX. The diagonal of H records how large each input channel usually is; the off-diagonal entries record whether two channels often appear together.

Then pick scales block by block. The candidates are the default scale code shifted a few steps up and down, with codes outside the format's legal range removed. Each candidate quantizes the block once, giving that block's error vector e. The score is eᵀH_b e, where H_b is the small part of H that belongs to this block's input channels; it measures how far this block's error alone would move the layer output. If H_b were replaced by the identity matrix, the score would fall back to mean squared error.

Once a block is chosen, its error is not just left there. Following GPTQ, SoScale uses the inverse of H to convert the block's error into adjustments of the weights not yet quantized, so later blocks compensate for it. The next block's scale is chosen on the compensated weights.

Each format searches its own codes. MXFP4 searches the power-of-two scale; NVFP4 searches the E4M3 scale shared by every 16 numbers while keeping the tensor-wide FP32 scale fixed; HiF4 searches the top-level E6M2 code and derives the two lower E1 scales from it.

Walking through it: the weight block in Figure 1

The example uses the Qwen3-8B weight block from Figure 1 of the paper; the scales 0.125 and 0.0625 come from the paper, and the individual element values are illustrative.

Step 1: the default rule. The largest value in the block is an outlier around −0.4, and the default scale is set to 0.125, so the outermost possible values are ±0.75 and neighboring grid points near 0 are 0.0625 apart. The outlier −0.4 rounds to −0.375, and no value at all lands on ±0.75. Most of the numbers crowded near 0 can only land on three grid points: 0 and ±0.0625.

Step 2: list the candidates. Shifting the scale code two steps up and down gives candidate scales 0.03125, 0.0625, 0.125, 0.25 and 0.5.

Step 3: try 0.0625. The possible values become 0, 0.03125, 0.0625, 0.09375, 0.125, 0.1875, 0.25 and 0.375. The outlier −0.4 is clipped to −0.375, the same value it landed on before, so its error is unchanged. Near 0 there are two extra grid points, ±0.03125. Illustratively, a weight of 0.03 could only become 0 under the default, with error 0.03; now it becomes 0.03125, with error about 0.001. The error curve in Figure 1 of the paper looks like this: the outlier's error is unchanged, and for a large group of values near 0 the error drops from about 0.03 to nearly 0.

Step 4: score. Each candidate's error vector e gets its eᵀH_b e, and errors that fall on frequently active input channels are penalized more. Mean squared error looks only at how big the errors are, so the two scores can pick different scales; SoScale picks the one with the lowest eᵀH_b e.

Step 5: compensate. The error left after quantizing this block is converted through the inverse of H onto the later columns, changing the not-yet-quantized weights in the working copy. The next block's candidates are then scored on these changed weights.

Element-wise errors of the 32 elements of the same weight block after sorting. The blue line is the default scale 0.125 and the orange line the searched 0.0625; the two lines meet at the element with the largest error, a dozen or so elements see their error drop sharply, and the rest almost coincide. Source: paper Figure 1
Element-wise errors of the 32 elements of the same weight block after sorting. The blue line is the default scale 0.125 and the orange line the searched 0.0625; the two lines meet at the element with the largest error, a dozen or so elements see their error drop sharply, and the rest almost coincide. Source: paper Figure 1

Results

Experiments run W4A4 on Qwen3-8B and Qwen3-14B: the weights and activations of the linear layers in the Transformer blocks are quantized to FP4, while embeddings, the output layer and normalization layers stay in BF16. Activations use a lightweight rule that computes a scale online for each token. Evaluation uses average accuracy over 7 downstream tasks.

Across two models and three formats, 6 groups in all, the best average score in every group goes to SoScale or SoScale with block Hadamard rotation. It beats the strongest non-SoScale method in each group by 0.21 to 0.77 percentage points and ScaleSearch by 0.47 to 1.10. The gain is largest under MXFP4. The paper's explanation is that, because its scale can only move in factors of 2, outliers most easily stretch the grid too coarse.

To separate what search and compensation each contribute, the paper ran an ablation on Qwen3-8B:

FormatDefaultScaleSearchMSE + compensationSoScale
MXFP469.7170.5670.6170.92
NVFP472.6272.8172.7073.56
HiF472.1172.4272.6372.95

(Numbers from Table 4 of the paper, average accuracy over 7 tasks, in %)

Adding error compensation to mean-squared-error search gives inconsistent results, and on NVFP4 it even lowers the score. With Hessian scoring plus compensation, all three formats are best. This shows that bolting on compensation is not enough; in this ablation, switching the score to the layer output is the step that helps consistently.

Block Hadamard rotation alone gives mixed results and is sometimes worse than default quantization. Combined with SoScale, it gives the best score for both models under MXFP4; but for Qwen3-8B under NVFP4 and Qwen3-14B under HiF4, SoScale alone is better. The paper also tested a version where activations use plain default quantization; SoScale's weight search still beats ScaleSearch's weight search in all 6 groups, by 0.08 to 1.92 percentage points.

What this means for people deploying models

If your hardware already supports MXFP4, NVFP4 or HiF4, SoScale's output stays in the original format and the inference kernel needs no change for it; one offline run gets you this accuracy.

What is stored is still standard FP4 values plus legal scale codes, and inference no longer searches weight scales. All the weight-side computation happens offline during quantization: H for every layer, quantizing each block with several candidates and computing eᵀH_b e for each, then error compensation. In the main experiments, activations still pick a scale per token with the online rule; the version that quantizes activations with plain default quantization skips even that.

If you want to try it, start with MXFP4, where the gain is largest and where adding Hadamard rotation is most worthwhile. Teams already using GPTQ only need to change the scale-picking step: score the candidates with eᵀH_b e on the compensated weights and keep the compensation as it is. Table 4 is also a reminder that simply attaching GPTQ compensation to mean-squared-error search gives unstable gains.

Limitations

The experiments cover only the 8B and 14B sizes of Qwen3, with WikiText as the only calibration set. Most gains are a few tenths of a percentage point, and NVFP4 and HiF4 are already close to BF16, leaving little room. SoScale searches only weight scales, activations still use a simple online rule, and the two are not optimized jointly.

How did this read for you?

One tap, no sign-in; tap again to undo. Only anonymous counts are kept, never who you are.

Discussion

Add experiment details, reproduction notes or a different take. Requires a GitHub sign-in.

Loading the discussion…