sgl-project/sglang · error · ValueError

weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

Error message

weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

What it means

This error is raised by the constructor of the ModelSlim W4A8 MXFP4 MoE quantization scheme in SGLang. The class handles exactly two weight matrices in a fused MoE layer: 'w13' (the fused gate/up projection) and 'w2' (the down projection). Passing any other prefix string means the caller constructed the quantizer for a layer it cannot handle.

Source

Thrown at python/sglang/srt/layers/quantization/modelslim/schemes/modelslim_w4a8_mxfp4_moe.py:30

from sglang.srt.layers.quantization.modelslim.schemes import ModelSlimMoEScheme
from sglang.srt.utils import set_weight_attrs

W4A8_MXFP4_BLOCK_SIZE = 32
W4A8_MXFP4_PACK_FACTOR = 2

__all__ = ["ModelSlimW4A8MXFP4MoE"]


class ModelSlimW4A8MXFP4MoE(ModelSlimMoEScheme):
    """Create one ModelSlim W4A8 MXFP expert-weight group (w13 or w2)."""

    def __init__(
        self,
        quant_config: Dict[str, Any],
        weight_prefix: str,
    ) -> None:
        if weight_prefix not in ("w13", "w2"):
            raise ValueError(
                f"weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'"
            )
        self.quant_config = quant_config
        self.weight_prefix = weight_prefix
        self.kernel = NPUW4A8MXFP4MoEMethod()

    def create_weights(
        self,
        layer: torch.nn.Module,
        num_experts: int,
        hidden_size: int,
        intermediate_size_per_partition: int,
        **extra_weight_attrs,
    ) -> None:
        from sglang.srt.layers.moe.fused_moe_triton import FusedMoeWeightScaleSupported

        extra_weight_attrs.update(
            {"quant_method": FusedMoeWeightScaleSupported.BLOCK.value}

View on GitHub (pinned to 0132848349)

Solutions

1. Check the call site that constructs the quantizer and pass exactly 'w13' for the gate/up projection or 'w2' for the down projection
2. If you are adding support for a new projection name, extend the tuple ('w13', 'w2') in the check AND make sure the downstream kernel actually supports it
3. Verify the model's MoE implementation maps its layer names onto the FusedMoE w13/w2 convention before wiring in ModelSlim

Example fix

// before
quant = ModelSlimW4A8MXFP4MoEQuantizer(cfg, weight_prefix="gate_up")
// after
quant = ModelSlimW4A8MXFP4MoEQuantizer(cfg, weight_prefix="w13")

allowed = ("w13", "w2")
if prefix not in allowed:
    raise ValueError(f"bad prefix {prefix!r}; expected one of {allowed}")
quant = ModelSlimW4A8MXFP4MoEQuantizer(cfg, weight_prefix=prefix)

from typing import Literal
WeightPrefix = Literal["w13", "w2"]

def is_weight_prefix(v: str) -> TypeGuard[WeightPrefix]:
    return v in ("w13", "w2")

When it happens

Trigger: Instantiating ModelSlimW4A8MXFP4MoEQuantizer (or the config class that wraps it) with a weight_prefix argument that is not exactly 'w13' or 'w2' — e.g. passing 'w1', 'gate_up_proj', 'w3', or a typo like 'W13'.

Common situations: Happens when porting a new MoE model to SGLang's NPU ModelSlim path and naming the layer prefix inconsistently with the fused-MoE convention, or when adapting an existing quant scheme class into a custom one and forgetting to update the allowed prefixes.

Related errors

• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• Unsupported ascend_dispatcher_output_dtype: {self.ascend_dis
• The W8A8Int8 Fused MoE scheme is implemented only for NPU fo
• The W4A8Int8 Fused MoE scheme is implemented only for NPU fo
• No ModelSlim MoE scheme found for layer {prefix}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1c480f0fa8ec333e. Report an issue: GitHub.