sgl-project/sglang · error · ValueError

Unsupported weight_bits: {weight_bits}, currently only suppo

Error message

Unsupported weight_bits: {weight_bits}, currently only support  {self.SUPPORTED_BITS}

What it means

The AutoRound quantization config only accepts certain weight bit-widths (AutoRoundConfig.SUPPORTED_BITS, effectively 4/8-bit INT). Passing any other weight_bits (e.g. 3, 16) when constructing the config raises immediately.

Source

Thrown at python/sglang/srt/layers/quantization/auto_round.py:74

        weight_bits: int,
        group_size: int,
        sym: bool = True,
        packing_format: str = "auto_round:auto_gptq",
        block_name_to_quantize: Optional[Union[str, list[str]]] = None,
        extra_config: Optional[dict[str, Any]] = None,
        data_type: str = "int",
        backend: str = "auto",
        lm_head_quantized: bool = False,
        desc_act: bool = False,
        dynamic: Optional[dict[str, dict[str, Union[int, bool]]]] = None,
        checkpoint_format: str = "",
        true_sequential: bool = False,
        static_groups: bool = False,
        gptq_defaulted_config_keys: Optional[tuple[str, ...]] = None,
    ) -> None:
        super().__init__()
        if weight_bits not in self.SUPPORTED_BITS:
            raise ValueError(
                f"Unsupported weight_bits: {weight_bits}, "
                f"currently only support  {self.SUPPORTED_BITS}"
            )
        if data_type not in self.SUPPORTED_DTYPES:
            raise ValueError(
                f"Unsupported data_type: {data_type},"
                f" currently only support  {self.SUPPORTED_DTYPES}"
            )
        if packing_format not in self.SUPPORTED_FORMATS:
            raise ValueError(
                f"Unsupported packing_format: {packing_format}, "
                f"currently only support  {self.SUPPORTED_FORMATS}"
            )
        if backend not in self.SUPPORTED_BACKENDS:
            raise ValueError(
                f"Unsupported backend: {backend},  "
                f"currently only support  {self.SUPPORTED_BACKENDS}"
            )

View on GitHub (pinned to 0132848349)

Solutions

  1. Re-export or pick a checkpoint with weight_bits 4 or 8
  2. Edit the config.json quantization_config bits to a supported value if it was hand-modified
  3. If you need other bit-widths, use a different quant method (e.g. gptq variants that support them)

Example fix

// before (config.json)
"quantization_config": {"bits": 3, ...}
// after
"quantization_config": {"bits": 4, ...}
Defensive patterns

Strategy: validation

Validate before calling

bits = quant_cfg.get("weight_bits", quant_cfg.get("bits"))
assert bits in AutoRoundConfig.SUPPORTED_BITS, f"bad bits {bits}"

Type guard

def is_supported_bits(bits: int) -> bool:
    return bits in AutoRoundConfig.SUPPORTED_BITS

Try / catch

try:
    cfg = AutoRoundConfig.from_config(quant_cfg)
except ValueError as e:
    if "weight_bits" in str(e): reexport_with_4_or_8bit()
    raise

Prevention

When it happens

Trigger: Creating AutoRoundConfig(weight_bits=...) or loading an AutoRound checkpoint whose quant_config in config.json specifies an unsupported bits value; also setting bits in an export script and then loading in SGLang.

Common situations: Exporting a 3-bit or 2-bit AutoRound model with the AutoRound tool then serving it with SGLang, or hand-editing quantization_config fields.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e9b9f46048b3d35d. Report an issue: GitHub.