sgl-project/sglang · error · ValueError

weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

Error message

weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

What it means

Same fused-MoE prefix contract as its W4A8 sibling: the ModelSlim W8A8 INT8 MoE quantizer is constructed per weight matrix and only accepts 'w13' (gate/up) or 'w2' (down). The check runs after the fields are assigned, so the object is partially initialized when it raises.

Source

Thrown at python/sglang/srt/layers/quantization/modelslim/schemes/modelslim_w8a8_int8_moe.py:40

    """
    W8A8 integer MoE scheme that creates weights for either the
    w13 (gate+up) or w2 (down) projection group.

    Two instances of this class are used per MoE layer:
      - weight_prefix="w13"   → handles the fused gate_proj + up_proj weights
      - weight_prefix="w2"    → handles the down_proj weights
    """

    def __init__(
        self,
        quant_config: Dict[str, Any],
        weight_prefix: str,  # "w13" or "w2"
    ) -> None:
        self.quant_config = quant_config
        self.kernel = NPUW8A8Int8MoEMethod()
        self.weight_prefix = weight_prefix
        if weight_prefix not in ("w13", "w2"):
            raise ValueError(
                f"weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'"
            )

    def create_weights(
        self,
        layer: torch.nn.Module,
        num_experts: int,
        hidden_size: int,
        intermediate_size_per_partition: int,
        **extra_weight_attrs,
    ) -> None:
        from sglang.srt.layers.moe.fused_moe_triton import FusedMoeWeightScaleSupported

        self.num_experts = num_experts
        extra_weight_attrs.update(
            {"quant_method": FusedMoeWeightScaleSupported.CHANNEL.value}
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Pass exactly 'w13' or 'w2' at the construction site
2. Align the new model's MoE weight names with SGLang's FusedMoE w13/w2 layout
3. Extend the validation tuple only if you also implement the kernel path for the new prefix

Example fix

// before
quant = ModelSlimW8A8Int8MoEQuantizer(cfg, weight_prefix="proj")
// after
quant = ModelSlimW8A8Int8MoEQuantizer(cfg, weight_prefix="w2")

if weight_prefix not in ("w13", "w2"):
    raise ValueError(f"unsupported prefix {weight_prefix!r}")
quant = ModelSlimW8A8Int8MoEQuantizer(cfg, weight_prefix=weight_prefix)

from typing import Literal
WeightPrefix = Literal["w13", "w2"]

def is_weight_prefix(v: str) -> TypeGuard[WeightPrefix]:
    return v in ("w13", "w2")

When it happens

Trigger: Constructing the W8A8 INT8 MoE quantizer with a weight_prefix other than 'w13'/'w2' — custom MoE implementations, renamed prefixes, or copy-paste from a non-fused linear quant scheme.

Common situations: Adding a new NPU-quantized MoE model whose layer naming doesn't follow the w13/w2 fused convention; adapting the class for a non-MoE linear layer where prefixes like 'weight' are used.

Related errors

• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'
• Unsupported ascend_dispatcher_output_dtype: {self.ascend_dis
• The W8A8Int8 Fused MoE scheme is implemented only for NPU fo
• The W4A8Int8 Fused MoE scheme is implemented only for NPU fo
• No ModelSlim MoE scheme found for layer {prefix}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d738fa2fa711075c. Report an issue: GitHub.