{"record":{"id":"d738fa2fa711075c","repo":"sgl-project/sglang","slug":"weight-prefix-must-be-w13-or-w2-got-weight-d738fa","errorCode":null,"errorMessage":"weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'","messagePattern":"weight_prefix must be 'w13' or 'w2', got '(.+?)'","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/modelslim/schemes/modelslim_w8a8_int8_moe.py","lineNumber":40,"sourceCode":"    \"\"\"\n    W8A8 integer MoE scheme that creates weights for either the\n    w13 (gate+up) or w2 (down) projection group.\n\n    Two instances of this class are used per MoE layer:\n      - weight_prefix=\"w13\"   → handles the fused gate_proj + up_proj weights\n      - weight_prefix=\"w2\"    → handles the down_proj weights\n    \"\"\"\n\n    def __init__(\n        self,\n        quant_config: Dict[str, Any],\n        weight_prefix: str,  # \"w13\" or \"w2\"\n    ) -> None:\n        self.quant_config = quant_config\n        self.kernel = NPUW8A8Int8MoEMethod()\n        self.weight_prefix = weight_prefix\n        if weight_prefix not in (\"w13\", \"w2\"):\n            raise ValueError(\n                f\"weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'\"\n            )\n\n    def create_weights(\n        self,\n        layer: torch.nn.Module,\n        num_experts: int,\n        hidden_size: int,\n        intermediate_size_per_partition: int,\n        **extra_weight_attrs,\n    ) -> None:\n        from sglang.srt.layers.moe.fused_moe_triton import FusedMoeWeightScaleSupported\n\n        self.num_experts = num_experts\n        extra_weight_attrs.update(\n            {\"quant_method\": FusedMoeWeightScaleSupported.CHANNEL.value}\n        )\n","sourceCodeStart":22,"sourceCodeEnd":58,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/modelslim/schemes/modelslim_w8a8_int8_moe.py#L22-L58","documentation":"Same fused-MoE prefix contract as its W4A8 sibling: the ModelSlim W8A8 INT8 MoE quantizer is constructed per weight matrix and only accepts 'w13' (gate/up) or 'w2' (down). The check runs after the fields are assigned, so the object is partially initialized when it raises.","triggerScenarios":"Constructing the W8A8 INT8 MoE quantizer with a weight_prefix other than 'w13'/'w2' — custom MoE implementations, renamed prefixes, or copy-paste from a non-fused linear quant scheme.","commonSituations":"Adding a new NPU-quantized MoE model whose layer naming doesn't follow the w13/w2 fused convention; adapting the class for a non-MoE linear layer where prefixes like 'weight' are used.","solutions":["Pass exactly 'w13' or 'w2' at the construction site","Align the new model's MoE weight names with SGLang's FusedMoE w13/w2 layout","Extend the validation tuple only if you also implement the kernel path for the new prefix"],"exampleFix":"// before\nquant = ModelSlimW8A8Int8MoEQuantizer(cfg, weight_prefix=\"proj\")\n// after\nquant = ModelSlimW8A8Int8MoEQuantizer(cfg, weight_prefix=\"w2\")","handlingStrategy":"validation","validationCode":"if weight_prefix not in (\"w13\", \"w2\"):\n    raise ValueError(f\"unsupported prefix {weight_prefix!r}\")\nquant = ModelSlimW8A8Int8MoEQuantizer(cfg, weight_prefix=weight_prefix)","typeGuard":"from typing import Literal\nWeightPrefix = Literal[\"w13\", \"w2\"]\n\ndef is_weight_prefix(v: str) -> TypeGuard[WeightPrefix]:\n    return v in (\"w13\", \"w2\")","tryCatchPattern":null,"preventionTips":["Keep the w13/w2 naming when porting new MoE models","Add a unit test asserting constructors reject unknown prefixes early"],"tags":["quantization","moe","npu","modelslim","validation"],"backgroundTag":"invalid-enum-argument","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}