sgl-project/sglang · error · ValueError

weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

Error message

weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

What it means

The ModelSlim W4A4 INT4 MoE scheme validates weight_prefix at construction and only accepts 'w13' (fused gate/up) or 'w2' (down projection). Any other value raises before any weights are created. Note this check runs after several attributes are already assigned, but the outcome is the same immediate ValueError.

Source

Thrown at python/sglang/srt/layers/quantization/modelslim/schemes/modelslim_w4a4_int4_moe.py:44

    Two instances of this class are used per MoE layer:
      - weight_prefix="w13"   → handles the fused gate_proj + up_proj weights
      - weight_prefix="w2"    → handles the down_proj weights
    """

    def __init__(
        self,
        quant_config: Dict[str, Any],
        weight_prefix: str,  # "w13" or "w2"
        group_size: int = 0,
    ) -> None:
        self.quant_config = quant_config
        self.kernel = NPUW4A4Int4MoEMethod()
        self.weight_prefix = weight_prefix
        self.group_size = group_size
        self.is_per_channel_weight = group_size == 0
        if weight_prefix not in ("w13", "w2"):
            raise ValueError(
                f"weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'"
            )

    def create_weights(
        self,
        layer: torch.nn.Module,
        num_experts: int,
        hidden_size: int,
        intermediate_size_per_partition: int,
        **extra_weight_attrs,
    ) -> None:
        """
        Create and register weight, scale, and offset parameters for the layer.
        Shape depends on the W4A4 packing environment flag and whether the weight
        prefix is "w13" or "w2".
        """
        from sglang.srt.layers.moe.fused_moe_triton import FusedMoeWeightScaleSupported

View on GitHub (pinned to 0132848349)

Solutions

1. Use 'w13' or 'w2' exactly when constructing the scheme
2. Route scheme creation through get_moe_scheme/instantiate so the correct weight_group is passed
3. If your model genuinely needs a third group, extend the validation and the NPUW4A4Int4MoEMethod kernel to handle it

Example fix

# before
scheme = ModelSlimW4A4Int4MoEScheme(cfg, weight_prefix="gate_up")
# after
scheme = ModelSlimW4A4Int4MoEScheme(cfg, weight_prefix="w13")

assert weight_prefix in ("w13", "w2"), f"bad weight_prefix {weight_prefix!r}"
scheme = ModelSlimW4A4Int4MoEScheme(cfg, weight_prefix, group_size=group_size)

def is_valid_weight_prefix(v: str) -> bool:
    return isinstance(v, str) and v in ("w13", "w2")

try:
    scheme = ModelSlimW4A4Int4MoEScheme(cfg, weight_prefix, group_size)
except ValueError as e:
    if "weight_prefix must be" in str(e):
        raise TypeError(f"map {weight_prefix!r} to 'w13'/'w2' before construction")
    raise

When it happens

Trigger: Constructing ModelSlimW4A4Int4MoEScheme with weight_prefix not in ('w13','w2'); typically from custom instantiation code, a modified scheme map, or a fork that passes a different weight-group label.

Common situations: Integrating a new NPU MoE model with different weight naming; copy-pasting instantiation from another quant framework that uses 'gate_up'/'down' labels; refactors changing weight-group constants.

Related errors

• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• No ModelSlim MoE scheme found for layer {prefix}
• The Triton WNA16 MoE backend only supports symmetric INT4 gr

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/98ed233f1d47ec4a. Report an issue: GitHub.