sgl-project/sglang · error · ValueError

weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

Error message

weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

What it means

The ModelSlim W4A8 INT8 MoE scheme constructor validates its weight_prefix argument and only accepts 'w13' or 'w2'. Because group_size, tp_size, and activation clipping behavior depend on which weight group the scheme targets, an invalid prefix is rejected immediately.

Source

Thrown at python/sglang/srt/layers/quantization/modelslim/schemes/modelslim_w4a8_int8_moe.py:40

    Two instances of this class are created per MoE layer:
      - weight_prefix="w13"  → handles gate + up projections
      - weight_prefix="w2"   → handles down projection

    Configuration flags (``is_per_channel_weight``, ``activation_use_clip``)
    are passed to the underlying NPU kernel.
    """

    def __init__(
        self,
        quant_config: Dict[str, Any],
        weight_prefix: str,
        group_size: int = 0,
        tp_size: int = 1,
        activation_use_clip: bool = False,
    ) -> None:
        if weight_prefix not in ("w13", "w2"):
            raise ValueError(
                f"weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'"
            )
        self.quant_config = quant_config
        self.weight_prefix = weight_prefix
        self.group_size = group_size
        self.tp_size = tp_size
        self.is_per_channel_weight = group_size == 0
        self.activation_use_clip = activation_use_clip
        self.kernel = NPUW4A8Int8MoEMethod(
            is_per_channel_weight=self.is_per_channel_weight,
            activation_use_clip=self.activation_use_clip,
        )

    def create_weights(
        self,
        layer: torch.nn.Module,
        num_experts: int,
        hidden_size: int,

View on GitHub (pinned to 0132848349)

Solutions

1. Use exactly 'w13' for gate/up and 'w2' for down when constructing the scheme
2. Validate the value at the call site (assert weight_prefix in ('w13','w2')) if it comes from config or user input
3. Extend the allowed values plus kernel logic if a genuinely new weight group is required

Example fix

# before
scheme = ModelSlimW4A8Int8MoEScheme(cfg, weight_prefix="down")
# after
scheme = ModelSlimW4A8Int8MoEScheme(cfg, weight_prefix="w2")

assert weight_prefix in ("w13", "w2"), f"bad weight_prefix {weight_prefix!r}"
scheme = ModelSlimW4A8Int8MoEScheme(cfg, weight_prefix, group_size, tp_size, activation_use_clip)

def is_valid_weight_prefix(v: str) -> bool:
    return isinstance(v, str) and v in ("w13", "w2")

try:
    scheme = ModelSlimW4A8Int8MoEScheme(cfg, weight_prefix, group_size, tp_size, use_clip)
except ValueError as e:
    if "weight_prefix must be" in str(e):
        raise ValueError("pass 'w13' (gate/up) or 'w2' (down)")
    raise

When it happens

Trigger: Calling ModelSlimW4A8Int4MoEScheme(...) (INT8 MoE scheme __init__) with a weight_prefix outside ('w13','w2'); standard loading passes the correct value via instantiate(weight_group=...), so custom code or a patched scheme map triggers this.

Common situations: Reusing W4A8 instantiation snippets with a new model's naming; dynamic prefix derivation with typos; forks adding new weight groups without updating the validation.

Related errors

• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• No ModelSlim MoE scheme found for layer {prefix}
• The W8A8Int8 Fused MoE scheme is implemented only for NPU fo

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/979e4d0f051dfa63. Report an issue: GitHub.