sgl-project/sglang · error · ValueError

SGLang's AutoRound GPTQ loader supports desc_act=False only.

Error message

SGLang's AutoRound GPTQ loader supports desc_act=False only. AutoRound auto_gptq export does not use act-order/desc_act=True; if this checkpoint is a GPTQModel act-order checkpoint, use `--quantization gptq` or `--quantization gptq_marlin` instead.

What it means

SGLang's AutoRound GPTQ loader only supports desc_act=False (no activation-order reordering), matching what AutoRound's auto_gptq export produces. A checkpoint with desc_act=True is rejected with guidance to use the gptq/gptq_marlin loaders, which do support act-order.

Source

Thrown at python/sglang/srt/layers/quantization/auto_round.py:306

        if not _is_cpu_amx_available:
            raise ValueError(_CPU_AMX_REQUIRED_MSG)

    def log_gptq_default_assumptions_once(self) -> None:
        if self._logged_gptq_default_assumptions or not self.gptq_defaulted_config_keys:
            return
        self._logged_gptq_default_assumptions = True
        default_summary = {
            key: _GPTQ_DEFAULTS[key] for key in self.gptq_defaulted_config_keys
        }
        logger.info(
            "AutoRound GPTQ config does not specify %s; using SGLang defaults %s.",
            ", ".join(self.gptq_defaulted_config_keys),
            default_summary,
        )

    def check_gptq_support(self) -> None:
        if self.desc_act:
            raise ValueError(
                "SGLang's AutoRound GPTQ loader supports desc_act=False only. "
                "AutoRound auto_gptq export does not use act-order/desc_act=True; "
                "if this checkpoint is a GPTQModel act-order checkpoint, use "
                "`--quantization gptq` or `--quantization gptq_marlin` instead."
            )

    def get_gptq_config_kwargs(
        self, weight_bits: int, group_size: int
    ) -> dict[str, Any]:
        self.log_gptq_default_assumptions_once()
        self.check_gptq_support()
        return {
            "weight_bits": weight_bits,
            "group_size": group_size,
            "lm_head_quantized": self.lm_head_quantized,
            "desc_act": self.desc_act,
            "dynamic": self.dynamic,
            "checkpoint_format": self.checkpoint_format,

View on GitHub (pinned to 0132848349)

Solutions

Serve the checkpoint with --quantization gptq or --quantization gptq_marlin as the message suggests
Set desc_act=false in the checkpoint's quantization_config only if the weights truly are not act-order reordered (AutoRound exports)
Re-export from AutoRound using its default auto_gptq format

Example fix

# before
python -m sglang.launch_server --model gptq-actorder-model --quantization auto_round
# after
python -m sglang.launch_server --model gptq-actorder-model --quantization gptq

Defensive patterns

Strategy: fallback

Validate before calling

if quant_method == "auto_round" and quant_cfg.get("desc_act", False):
    quant_method = "gptq_marlin"  # act-order capable loader

Type guard

def is_autoround_compatible(desc_act: bool) -> bool:
    return not desc_act

Try / catch

try:
    launch(quant="auto_round")
except ValueError as e:
    if "desc_act=False only" in str(e):
        launch(quant="gptq")  # fallback loader

Prevention

Inspect desc_act in quantization_config before choosing the quant flag
Default to gptq/gptq_marlin for GPTQModel act-order checkpoints

When it happens

Trigger: Loading a GPTQModel-style act-order checkpoint through --quantization auto_round; check_gptq_support fires while building GPTQ config kwargs during layer quantization setup.

Common situations: Downloading a community GPTQ act-order checkpoint and forcing auto_round, or AutoRound-exported models whose config was later edited to desc_act=true.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c23a5d36dfa5693d. Report an issue: GitHub.