{"record":{"id":"c23a5d36dfa5693d","repo":"sgl-project/sglang","slug":"sglang-s-autoround-gptq-loader-supports-desc-act-f","errorCode":null,"errorMessage":"SGLang's AutoRound GPTQ loader supports desc_act=False only. AutoRound auto_gptq export does not use act-order/desc_act=True; if this checkpoint is a GPTQModel act-order checkpoint, use `--quantization gptq` or `--quantization gptq_marlin` instead.","messagePattern":"SGLang's AutoRound GPTQ loader supports desc_act=False only\\. AutoRound auto_gptq export does not use act-order/desc_act=True; if this checkpoint is a GPTQModel act-order checkpoint, use `--quantization gptq` or `--quantization gptq_marlin` instead\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/auto_round.py","lineNumber":306,"sourceCode":"        if not _is_cpu_amx_available:\n            raise ValueError(_CPU_AMX_REQUIRED_MSG)\n\n    def log_gptq_default_assumptions_once(self) -> None:\n        if self._logged_gptq_default_assumptions or not self.gptq_defaulted_config_keys:\n            return\n        self._logged_gptq_default_assumptions = True\n        default_summary = {\n            key: _GPTQ_DEFAULTS[key] for key in self.gptq_defaulted_config_keys\n        }\n        logger.info(\n            \"AutoRound GPTQ config does not specify %s; using SGLang defaults %s.\",\n            \", \".join(self.gptq_defaulted_config_keys),\n            default_summary,\n        )\n\n    def check_gptq_support(self) -> None:\n        if self.desc_act:\n            raise ValueError(\n                \"SGLang's AutoRound GPTQ loader supports desc_act=False only. \"\n                \"AutoRound auto_gptq export does not use act-order/desc_act=True; \"\n                \"if this checkpoint is a GPTQModel act-order checkpoint, use \"\n                \"`--quantization gptq` or `--quantization gptq_marlin` instead.\"\n            )\n\n    def get_gptq_config_kwargs(\n        self, weight_bits: int, group_size: int\n    ) -> dict[str, Any]:\n        self.log_gptq_default_assumptions_once()\n        self.check_gptq_support()\n        return {\n            \"weight_bits\": weight_bits,\n            \"group_size\": group_size,\n            \"lm_head_quantized\": self.lm_head_quantized,\n            \"desc_act\": self.desc_act,\n            \"dynamic\": self.dynamic,\n            \"checkpoint_format\": self.checkpoint_format,","sourceCodeStart":288,"sourceCodeEnd":324,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/auto_round.py#L288-L324","documentation":"SGLang's AutoRound GPTQ loader only supports desc_act=False (no activation-order reordering), matching what AutoRound's auto_gptq export produces. A checkpoint with desc_act=True is rejected with guidance to use the gptq/gptq_marlin loaders, which do support act-order.","triggerScenarios":"Loading a GPTQModel-style act-order checkpoint through --quantization auto_round; check_gptq_support fires while building GPTQ config kwargs during layer quantization setup.","commonSituations":"Downloading a community GPTQ act-order checkpoint and forcing auto_round, or AutoRound-exported models whose config was later edited to desc_act=true.","solutions":["Serve the checkpoint with --quantization gptq or --quantization gptq_marlin as the message suggests","Set desc_act=false in the checkpoint's quantization_config only if the weights truly are not act-order reordered (AutoRound exports)","Re-export from AutoRound using its default auto_gptq format"],"exampleFix":"# before\npython -m sglang.launch_server --model gptq-actorder-model --quantization auto_round\n# after\npython -m sglang.launch_server --model gptq-actorder-model --quantization gptq","handlingStrategy":"fallback","validationCode":"if quant_method == \"auto_round\" and quant_cfg.get(\"desc_act\", False):\n    quant_method = \"gptq_marlin\"  # act-order capable loader","typeGuard":"def is_autoround_compatible(desc_act: bool) -> bool:\n    return not desc_act","tryCatchPattern":"try:\n    launch(quant=\"auto_round\")\nexcept ValueError as e:\n    if \"desc_act=False only\" in str(e):\n        launch(quant=\"gptq\")  # fallback loader","preventionTips":["Inspect desc_act in quantization_config before choosing the quant flag","Default to gptq/gptq_marlin for GPTQModel act-order checkpoints"],"tags":["quantization","auto-round","gptq","desc-act"],"backgroundTag":"unsupported-checkpoint-format","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}