{"record":{"id":"b145fb29cf07d908","repo":"sgl-project/sglang","slug":"unsupported-num-bits-num-bits-supported-num-b","errorCode":null,"errorMessage":"Unsupported num_bits = {num_bits}. Supported num_bits = {WNA16_SUPPORTED_TYPES_MAP.keys()}","messagePattern":"Unsupported num_bits = (.+?)\\. Supported num_bits = (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_wNa16.py","lineNumber":85,"sourceCode":"                 strategy: str,\n                 num_bits: int,\n                 group_size: Optional[int] = None,\n                 symmetric: Optional[bool] = True,\n                 actorder: Optional[ActivationOrdering] = None):\n\n        self.pack_factor = 32 // num_bits\n        self.strategy = strategy\n        self.symmetric = symmetric\n        self.group_size = -1 if group_size is None else group_size\n        self.has_g_idx = actorder == ActivationOrdering.GROUP\n\n        if self.group_size == -1 and self.strategy != \"channel\":\n            raise ValueError(\"Marlin kernels require group quantization or \"\n                             \"channelwise quantization, but found no group \"\n                             \"size and strategy is not channelwise.\")\n\n        if num_bits not in WNA16_SUPPORTED_TYPES_MAP:\n            raise ValueError(\n                f\"Unsupported num_bits = {num_bits}. \"\n                f\"Supported num_bits = {WNA16_SUPPORTED_TYPES_MAP.keys()}\")\n\n        self.quant_type = (WNA16_ZP_SUPPORTED_TYPES_MAP[num_bits]\n                           if not self.symmetric else\n                           WNA16_SUPPORTED_TYPES_MAP[num_bits])\n\n    @classmethod\n    def get_min_capability(cls) -> int:\n        # ampere and up\n        return 80\n\n    def create_weights(self, layer: torch.nn.Module, output_size: int,\n                       input_size: int, output_partition_sizes: list[int],\n                       input_size_per_partition: int,\n                       params_dtype: torch.dtype, weight_loader: Callable,\n                       **kwargs):\n","sourceCodeStart":67,"sourceCodeEnd":103,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_wNa16.py#L67-L103","documentation":"The wNa16 Marlin scheme only supports specific bit-widths listed in WNA16_SUPPORTED_TYPES_MAP (typically 4 and 8). num_bits outside that set (e.g. 2, 3, 6) has no kernel dtype mapping and is rejected at init.","triggerScenarios":"A compressed-tensors checkpoint with weights.num_bits not in the supported set (commonly anything other than 4/8) initializing CompressedTensorsWNA16.","commonSituations":"Exotic bit-widths from experimental quantizers; AWQ/GPTQ variants with 3-bit weights.","solutions":["Re-quantize to a supported bit width (4 or 8)","Check the error's printed supported keys against your config","Upgrade SGLang if wider bit support was added upstream"],"exampleFix":"// before\n\"weights\": {\"num_bits\": 3, \"strategy\": \"group\", \"group_size\": 128}\n// after\n\"weights\": {\"num_bits\": 4, \"strategy\": \"group\", \"group_size\": 128}","handlingStrategy":"type-guard","validationCode":null,"typeGuard":"SUPPORTED = {4, 8}\ndef num_bits_ok(w):\n    return w[\"num_bits\"] in SUPPORTED","tryCatchPattern":null,"preventionTips":["Restrict quantization bit-widths to 4 or 8 for compressed-tensors wNa16"],"tags":["quantization","marlin","bit-width","unsupported-format"],"backgroundTag":"unsupported-quantization-format","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}