sgl-project/sglang · error · ValueError

weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

Error message

weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

What it means

ModelSlim MXFP8 MoE scheme constructor validates that weight_prefix is exactly 'w13' or 'w2' — the two weight groups of a fused MoE layer (fused gate/up vs down). Any other string (e.g. 'w1', 'w_gate_up', typo, or None-ish value) is rejected immediately at construction.

Source

Thrown at python/sglang/srt/layers/quantization/modelslim/schemes/modelslim_mxfp8_moe.py:49

    Offline MXFP8 MoE scheme that creates weights for either the
    w13 (gate+up) or w2 (down) projection group.

    Two instances of this class are used per MoE layer:
      - weight_prefix="w13"   → handles the fused gate_proj + up_proj weights
      - weight_prefix="w2"    → handles the down_proj weights

    The float8_e4m3fn weight dtype allocated here is what tells
    ``NPUMXFP8MoEMethod.process_weights_after_loading`` to take its offline
    (re-layout only) branch instead of quantising the weights itself.
    """

    def __init__(
        self,
        quant_config: Dict[str, Any],
        weight_prefix: str,  # "w13" or "w2"
    ) -> None:
        if weight_prefix not in ("w13", "w2"):
            raise ValueError(
                f"weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'"
            )
        self.quant_config = quant_config
        self.weight_prefix = weight_prefix
        self.kernel = NPUMXFP8MoEMethod(weight_prefix)

    def create_weights(
        self,
        layer: torch.nn.Module,
        num_experts: int,
        hidden_size: int,
        intermediate_size_per_partition: int,
        **extra_weight_attrs,
    ) -> None:
        from sglang.srt.layers.moe.fused_moe_triton import FusedMoeWeightScaleSupported

        self.num_experts = num_experts
        extra_weight_attrs.update(

View on GitHub (pinned to 0132848349)

Solutions

1. Pass exactly 'w13' for the fused gate/up scheme or 'w2' for the down projection
2. Prefer letting ModelSlimConfig.get_moe_scheme instantiate schemes rather than constructing them manually
3. If adding a new weight group, extend the constructor's allowed tuple and the kernel mapping (NPUMXFP8MoEMethod) accordingly

Example fix

# before
scheme = ModelSlimMXFP8MoEScheme(cfg, weight_prefix="w1")
# after
scheme = ModelSlimMXFP8MoEScheme(cfg, weight_prefix="w13")

if weight_prefix not in ("w13", "w2"):
    raise ValueError(f"bad weight_prefix {weight_prefix!r}; expected 'w13'/'w2'")
scheme = ModelSlimMXFP8MoEScheme(quant_config, weight_prefix)

def is_valid_weight_prefix(v: str) -> bool:
    return isinstance(v, str) and v in ("w13", "w2")

try:
    scheme = ModelSlimMXFP8MoEScheme(cfg, weight_prefix)
except ValueError as e:
    if "weight_prefix must be" in str(e):
        weight_prefix = "w13" if "gate" in weight_prefix else "w2"
        scheme = ModelSlimMXFP8MoEScheme(cfg, weight_prefix)
    else:
        raise

When it happens

Trigger: Constructing ModelSlimMXFP8MoEScheme(quant_config, weight_prefix) with a prefix not in ('w13','w2'); in normal loading this is supplied by get_moe_scheme via instantiate(..., weight_group=...), so user code or a custom scheme map passing a wrong constant triggers it.

Common situations: Custom MoE models or plugins instantiating the scheme directly with their own naming; refactors renaming weight groups; copy-paste from W4A8 code paths using different group labels.

Related errors

• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• No ModelSlim MoE scheme found for layer {prefix}
• native MXFP8 MoE only supports gated swiglu-oai, got {activa

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/08c8c385f59cf8b7. Report an issue: GitHub.