A mixture-of-experts checkpoint that will not fit on the card is mostly experts. In Qwen3-30B-A3B they are the great majority of the weights, so removing a quarter of them buys back a quarter of the model. The question is which quarter.

The obvious answer is: whichever ones the router hardly uses. In AMCT that criterion is called activation_count, and in plenty of cases it is what we recommend.

While working on expert pruning we ran into something more interesting than that. It is now shipping as a new criterion in AMCT's structured pruning module, and this post is about what it does and what it is worth.

Two signals from the router

Each token that reaches a MoE layer selects top_k experts and assigns a weight to each one. A calibration pass therefore gives us two signals for every expert: how often the router selected it and how much weight it received when selected.

The two signals do not always produce the same ranking.

Schematic, not measured. Both experts received the same total weight.

Expert A appears in eighteen tokens' top-8 and receives a weight of 0.11 each time. Expert B appears in ten, usually with a small weight, but receives more than half of the routing weight for three tokens. Their total weight is identical, while their selection counts are not.

amct_pytorch.pruning exposes both signals. You select the criterion in the configuration passed to amct.prune.

activation_count ranks experts by the number of selections. mass_variance can rank them by total routing mass or by the distribution of their routing weights. Its variance_score option selects conditional variance (cond, the default), the maximum observed weight (peak), or their product (cvxpeak).

The boundary option controls where the criterion changes. Layers up to the boundary use routing mass; later layers use variance. The default boundary is 10.

Why weight distribution matters

When an expert is pruned, its tokens are routed to the next expert in the ranking.

Replacing Expert A is relatively cheap. It contributes about a tenth of the routing weight whenever it appears, while seven other experts already handle the rest. The next-ranked expert can replace a small contribution.

Replacing Expert B is riskier. It carries more than half of the routing weight for three tokens, and the replacement expert was ranked much lower for those tokens. A selection count does not capture that difference.

This is the case mass_variance is designed to detect. An expert that is selected less often may still matter when its routing weights are concentrated on a small number of tokens.

Results on Qwen3-30B-A3B

We tested Qwen3-30B-A3B-Base on one Ascend 910B3. The model has 48 routed layers, 128 experts per layer, and top_k 8. For each criterion, we ranked the experts, disabled the bottom share by setting their router logits to negative infinity, and measured held-out perplexity. We did not run a recovery pass, so the comparison isolates the selection criterion.

We also included random selection as a control.

Perplexity above the unpruned model. Shorter is better.

criterion	general, 25% cut	general, 40% cut	maths, 25% cut
mass_variance, all variance	+19.0%	+53.2%	+0.8%
MOE_MASSVAR_PRUNE_CFG as shipped	+17.3%	+54.9%	+6.3%
mass_variance, all mass	+27.7%	+79.3%	+8.4%
activation_count	+51.7%	+84.2%	+8.4%
random	+52.0%	+174.6%	+18.8%

The pure criteria have the same ordering in all three conditions: variance performs better than total routing mass, and routing mass performs better than selection count.

At a 25% cut on general text, activation_count increases perplexity by 51.7%, almost the same as random selection at 52.0%. All-variance scoring limits the increase to 19.0%, while the shipped preset reaches 17.3%. On maths, all-variance scoring increases perplexity by 0.8%, compared with 8.4% for activation_count and 18.8% for random selection.

activation_count still carries useful information in other conditions. At a 40% cut it performs much better than random selection, and it is competitive on the smaller models discussed below. It is simply not the best criterion for this Qwen3-30B experiment.

We ran two checks on the variance result. First, conditional variance could be unstable for experts with very few selections. With 21,374 calibration tokens, however, the median layer has only 2 experts out of 128 with fewer than 30 selections. Second, variance could be acting as an inverse routing-mass score. The median rank correlation between the two scores is +0.53 on Qwen3-30B and +0.59 on Qwen3-VL-30B-A3B-Instruct. The scores usually agree, but variance performs better where their rankings differ on this model.

Results vary by model

The smaller models do not follow the Qwen3-30B result.

Expert and top-k counts are real. The arrangement is schematic.

On granite-1b-a400m, using routing mass in every layer gives the best result in the sweep at 94.8 perplexity. Switching only the deepest layer to variance raises perplexity to 1801.5. On OLMoE-1B-7B, switching that layer to variance improves perplexity from 46.9 to 35.3. On Qwen3-30B, all-variance scoring beats all-mass scoring at every pruning ratio we tested.

The same boundary setting therefore produces a different answer for each model.

One possible explanation is expert density. Granite routes 8 of 32 experts for each token, so one quarter of the layer is active. Qwen3-30B routes 8 of 128, leaving more room for experts that activate rarely but strongly.

Schematic. Colours represent possible expert specialisations.

Our hypothesis is that larger expert pools can separate roles more clearly and may retain more redundancy. Calibration can then identify experts that are rarely needed or overlap with others. In a smaller pool, each expert may carry several roles, which makes any removal more disruptive. This is a hypothesis rather than a measured explanation, but it is consistent with the difference between the models we tested.

This model dependence also exposes a problem with boundary="auto". The automatic mode compares the coefficient of variation of the two scores in each layer and selects whichever score is more spread out. On Qwen3-30B, it chooses variance in all 48 layers with general calibration text, but in only 14 of 48 layers with maths text. The boundary changes with the calibration set, even though the model has not changed. We therefore do not recommend the automatic boundary.

Using the criterion in AMCT

The implementation is available in amct_pytorch.pruning on the cann/amct master branch under Apache 2.0.

The smallest useful call supplies calibration data and an acceptable accuracy loss:

import amct_pytorch as amct

amct.prune(model, data=calib, tolerance=0.02)

The pruning ratio is an output. The tool evaluates a grid of candidate ratios and applies the largest cut that remains within the tolerance.

Because the best criterion depends on the model, MOE_VARIANCE_MENU_CFG can compare the available candidates:

from amct_pytorch.pruning import MOE_VARIANCE_MENU_CFG

amct.prune(model, MOE_VARIANCE_MENU_CFG,
           data=calib, eval_data=val, evaluator=ev)

Use separate calibration and evaluation data for this comparison. Variance contains more information than a selection count and has more opportunity to fit the calibration set. The menu keeps a candidate only when it beats the safe fallback on held-out data. If you skip the menu, start with MOE_MASSVAR_PRUNE_CFG rather than activation_count.

Three practical recommendations follow from the measurements:

Provide finetune_fn when possible. A short recovery pass usually increases the acceptable pruning ratio more than changing the selection criterion.

Prune before quantization. Pruning changes tensor shapes, so quantization calibration should run on the pruned model.

Run the random control at least once. It shows whether the criterion is adding useful information for your model and dataset.

New criteria can be added by implementing one class and registering it with the preset registry. Contributions are welcome.

References

• Structured pruning reference: https://gitcode.com/cann/amct/blob/master/amct_pytorch/pruning/README_en.md
• Qwen3.6-MoE on a single card: https://gitcode.com/cann/amct/blob/master/examples/models/qwen3.6/Qwen3.6-Moe-Pruning_en.md
• The criteria and presets: https://gitcode.com/cann/amct/blob/master/amct_pytorch/pruning/presets.py
• The ratio search: https://gitcode.com/cann/amct/blob/master/amct_pytorch/pruning/accuracy_based_auto_prune.py