{"record":{"id":"e9b9f46048b3d35d","repo":"sgl-project/sglang","slug":"unsupported-weight-bits-weight-bits-currently","errorCode":null,"errorMessage":"Unsupported weight_bits: {weight_bits}, currently only support  {self.SUPPORTED_BITS}","messagePattern":"Unsupported weight_bits: (.+?), currently only support  (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/auto_round.py","lineNumber":74,"sourceCode":"        weight_bits: int,\n        group_size: int,\n        sym: bool = True,\n        packing_format: str = \"auto_round:auto_gptq\",\n        block_name_to_quantize: Optional[Union[str, list[str]]] = None,\n        extra_config: Optional[dict[str, Any]] = None,\n        data_type: str = \"int\",\n        backend: str = \"auto\",\n        lm_head_quantized: bool = False,\n        desc_act: bool = False,\n        dynamic: Optional[dict[str, dict[str, Union[int, bool]]]] = None,\n        checkpoint_format: str = \"\",\n        true_sequential: bool = False,\n        static_groups: bool = False,\n        gptq_defaulted_config_keys: Optional[tuple[str, ...]] = None,\n    ) -> None:\n        super().__init__()\n        if weight_bits not in self.SUPPORTED_BITS:\n            raise ValueError(\n                f\"Unsupported weight_bits: {weight_bits}, \"\n                f\"currently only support  {self.SUPPORTED_BITS}\"\n            )\n        if data_type not in self.SUPPORTED_DTYPES:\n            raise ValueError(\n                f\"Unsupported data_type: {data_type},\"\n                f\" currently only support  {self.SUPPORTED_DTYPES}\"\n            )\n        if packing_format not in self.SUPPORTED_FORMATS:\n            raise ValueError(\n                f\"Unsupported packing_format: {packing_format}, \"\n                f\"currently only support  {self.SUPPORTED_FORMATS}\"\n            )\n        if backend not in self.SUPPORTED_BACKENDS:\n            raise ValueError(\n                f\"Unsupported backend: {backend},  \"\n                f\"currently only support  {self.SUPPORTED_BACKENDS}\"\n            )","sourceCodeStart":56,"sourceCodeEnd":92,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/auto_round.py#L56-L92","documentation":"The AutoRound quantization config only accepts certain weight bit-widths (AutoRoundConfig.SUPPORTED_BITS, effectively 4/8-bit INT). Passing any other weight_bits (e.g. 3, 16) when constructing the config raises immediately.","triggerScenarios":"Creating AutoRoundConfig(weight_bits=...) or loading an AutoRound checkpoint whose quant_config in config.json specifies an unsupported bits value; also setting bits in an export script and then loading in SGLang.","commonSituations":"Exporting a 3-bit or 2-bit AutoRound model with the AutoRound tool then serving it with SGLang, or hand-editing quantization_config fields.","solutions":["Re-export or pick a checkpoint with weight_bits 4 or 8","Edit the config.json quantization_config bits to a supported value if it was hand-modified","If you need other bit-widths, use a different quant method (e.g. gptq variants that support them)"],"exampleFix":"// before (config.json)\n\"quantization_config\": {\"bits\": 3, ...}\n// after\n\"quantization_config\": {\"bits\": 4, ...}","handlingStrategy":"validation","validationCode":"bits = quant_cfg.get(\"weight_bits\", quant_cfg.get(\"bits\"))\nassert bits in AutoRoundConfig.SUPPORTED_BITS, f\"bad bits {bits}\"","typeGuard":"def is_supported_bits(bits: int) -> bool:\n    return bits in AutoRoundConfig.SUPPORTED_BITS","tryCatchPattern":"try:\n    cfg = AutoRoundConfig.from_config(quant_cfg)\nexcept ValueError as e:\n    if \"weight_bits\" in str(e): reexport_with_4_or_8bit()\n    raise","preventionTips":["Validate checkpoint config.json fields before launch","Standardize on 4-bit exports for AutoRound"],"tags":["quantization","auto-round","weight-bits"],"backgroundTag":"invalid-configuration-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}