{"record":{"id":"1dfc094a382f0a9c","repo":"sgl-project/sglang","slug":"num-bits-must-be-4-or-8-got-1dfc09","errorCode":null,"errorMessage":"num_bits must be 4 or 8, got {}","messagePattern":"num_bits must be 4 or 8, got (.+?)","errorType":"validation","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/moe_wna16.py","lineNumber":61,"sourceCode":"        for block in [0, 1]:\n            for row in [\n                2 * (i % 4),\n                2 * (i % 4) + 1,\n                2 * (i % 4 + 4),\n                2 * (i % 4 + 4) + 1,\n            ]:\n                perm1.append(16 * row + col + 8 * block)\n        for j in range(4):\n            perm_list.extend([p + 256 * j for p in perm1])\n\n    perm = np.array(perm_list)\n\n    if num_bits == 4:\n        interleave = np.array([0, 2, 4, 6, 1, 3, 5, 7])\n    elif num_bits == 8:\n        interleave = np.array([0, 2, 1, 3])\n    else:\n        raise Exception(\"num_bits must be 4 or 8, got {}\".format(num_bits))\n\n    perm = perm.reshape((-1, len(interleave)))[:, interleave].ravel()\n    perm = torch.from_numpy(perm)\n    return perm\n\n\nclass MoeWNA16Config(QuantizationConfig):\n    \"\"\"Config class for MOE WNA16 (W8A16/W4A16) quantization.\"\"\"\n\n    def __init__(\n        self,\n        linear_quant_method: str,\n        weight_bits: int,\n        group_size: int,\n        has_zp: bool,\n        lm_head_quantized: bool,\n        modules_to_not_convert: Optional[List[str]],\n        full_config: Dict[str, Any],","sourceCodeStart":43,"sourceCodeEnd":79,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/moe_wna16.py#L43-L79","documentation":"get_weight_perm builds the marlin-style weight permutation for the moe_wna16 GPTQ/AWQ kernels. The interleave pattern differs for 4-bit (8-element interleave) and 8-bit (4-element interleave) weights; any other num_bits value has no defined permutation so the helper raises a bare Exception.","triggerScenarios":"Calling get_weight_perm with num_bits other than 4 or 8 — e.g. a wna16 config advertising 2-bit, 3-bit, or 16-bit quantization for a MoE layer.","commonSituations":"Loading a MoE checkpoint quantized to an unusual bit-width (w2a16/w3a16 variants), or hand-editing quant_config.json with bits: 2/3 and expecting the marlin path to handle it.","solutions":["Re-quantize or use a checkpoint with weight_bits set to 4 or 8 (gptq w4a16 / w8a16 MoE)","Fix quant_config.json's bits field if it was edited to an unsupported value","If experimenting, add an explicit interleave table entry for your bit-width and verify kernel correctness — do not bypass the check blindly"],"exampleFix":"// before\nperm = get_weight_perm(num_bits=3, ...)\n// after\nperm = get_weight_perm(num_bits=4, ...)\n// or re-quantize: python -m sglang.srt.quantization ... --bits 4","handlingStrategy":"validation","validationCode":"if num_bits not in (4, 8):\n    raise ValueError(f\"num_bits {num_bits} unsupported; use 4 or 8\")\nperm = get_weight_perm(num_bits, perm, sym_temp)","typeGuard":null,"tryCatchPattern":"try:\n    perm = get_weight_perm(num_bits, ...)\nexcept Exception as e:\n    if \"num_bits\" in str(e):\n        raise ValueError(\"Re-quantize the MoE to 4- or 8-bit weights\") from e\n    raise","preventionTips":["Validate weight_bits in quant_config.json before loading","Prefer standard w4a16/w8a16 GPTQ/AWQ MoE checkpoints"],"tags":["quantization","moe","gptq","awq","marlin"],"backgroundTag":"unsupported-quant-bit-width","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}