sgl-project/sglang · error · ValueError

weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

Error message

weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

What it means

The ModelSlim W4A4 MXFP4 MoE scheme constructor enforces that weight_prefix is 'w13' (fused gate/up projections) or 'w2' (down projection). It fails fast on any other label before building the NPUW4A4MXFP4MoEMethod kernel.

Source

Thrown at python/sglang/srt/layers/quantization/modelslim/schemes/modelslim_w4a4_mxfp4_moe.py:29

)
from sglang.srt.layers.quantization.modelslim.schemes import ModelSlimMoEScheme
from sglang.srt.utils import set_weight_attrs

MXFP4_BLOCK_SIZE = 32

__all__ = ["ModelSlimW4A4MXFP4MoE"]


class ModelSlimW4A4MXFP4MoE(ModelSlimMoEScheme):
    """Create one ModelSlim MXFP4 expert-weight group (w13 or w2)."""

    def __init__(
        self,
        quant_config: Dict[str, Any],
        weight_prefix: str,
    ) -> None:
        if weight_prefix not in ("w13", "w2"):
            raise ValueError(
                f"weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'"
            )
        self.quant_config = quant_config
        self.weight_prefix = weight_prefix
        self.kernel = NPUW4A4MXFP4MoEMethod()

    def create_weights(
        self,
        layer: torch.nn.Module,
        num_experts: int,
        hidden_size: int,
        intermediate_size_per_partition: int,
        **extra_weight_attrs,
    ) -> None:
        from sglang.srt.layers.moe.fused_moe_triton import FusedMoeWeightScaleSupported

        extra_weight_attrs.update(
            {"quant_method": FusedMoeWeightScaleSupported.BLOCK.value}

View on GitHub (pinned to 0132848349)

Solutions

1. Pass 'w13' or 'w2' verbatim (case-sensitive, no extra characters)
2. Log or assert the computed weight_prefix before construction if it is derived dynamically
3. Delegate instantiation to get_moe_scheme's scheme map instead of manual construction

Example fix

# before
scheme = ModelSlimW4A4MXFP4MoEScheme(cfg, weight_prefix="W13")
# after
scheme = ModelSlimW4A4MXFP4MoEScheme(cfg, weight_prefix="w13")

if weight_prefix not in ("w13", "w2"):
    raise ValueError("expected 'w13' or 'w2'")
scheme = ModelSlimW4A4MXFP4MoEScheme(quant_config, weight_prefix)

def is_valid_weight_prefix(v: str) -> bool:
    return isinstance(v, str) and v in ("w13", "w2")

try:
    scheme = ModelSlimW4A4MXFP4MoEScheme(cfg, weight_prefix)
except ValueError as e:
    if "weight_prefix must be" in str(e):
        logger.error("normalize weight_prefix to 'w13'/'w2' (case-sensitive)")
    raise

When it happens

Trigger: Instantiating ModelSlimW4A4MXFP4MoEScheme with an unexpected weight_prefix string; the stock loader always passes 'w13'/'w2' via get_moe_scheme, so this appears in custom or modified code paths.

Common situations: Custom MoE integrations, forks renaming weight groups, or glue code that derives weight_prefix dynamically and produces an off-by-one/typo like 'w12' or 'W13' (case-sensitive).

Related errors

• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• No ModelSlim MoE scheme found for layer {prefix}
• Mismatched ModelSlim quantization for W13 in layer {prefix}:

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fe0e616691037e12. Report an issue: GitHub.