sgl-project/sglang · error · ValueError

Currently, desc_act (True) is not supported by GPTQ quantiza

Error message

Currently, desc_act (True) is not supported by GPTQ quantization on npu.

What it means

The Ascend NPU GPTQ kernel (GPTQLinearAscendKernel.create_weights) rejects desc_act=True (activation-order-aware GPTQ, also called act-order) because the NPU kernel cannot reorder groups according to the g_idx permutation.

Source

Thrown at python/sglang/srt/layers/quantization/gptq/schemes/gptq_linear.py:164

        self.kernel.process_weights_after_loading(layer)

    def apply_weights(
        self, layer: torch.nn.Module, x: torch.Tensor, bias: Optional[torch.Tensor]
    ):
        return self.kernel.apply(layer, x, bias)


class GPTQAscendLinearScheme(GPTQLinearScheme):
    def _init_kernel(self, quant_config: GPTQConfig):
        from sglang.srt.hardware_backend.npu.quantization.gptq_kernels import (
            GPTQLinearAscendKernel,
        )

        return GPTQLinearAscendKernel(quant_config)

    def create_weights(self, layer: torch.nn.Module, **kwargs):
        if self.quant_config.desc_act:
            raise ValueError(
                "Currently, desc_act (True) is not supported by GPTQ "
                "quantization on npu."
            )

        super().create_weights(layer=layer, **kwargs)
        set_weight_attrs(layer.qzeros, {"pack_factor": self.quant_config.pack_factor})
        set_weight_attrs(layer.qweight, {"pack_factor": self.quant_config.pack_factor})


class GPTQXPULinearScheme(GPTQLinearScheme):
    def _init_kernel(self, quant_config: GPTQConfig):
        from sglang.srt.hardware_backend.xpu.quantization.gptq_kernels import (
            GPTQXPULinearKernel,
        )

        return GPTQXPULinearKernel(quant_config)

View on GitHub (pinned to 0132848349)

Solutions

1. Use a GPTQ checkpoint quantized with desc_act=false / act_order disabled
2. Edit quantization_config.json to set desc_act: false only if the checkpoint truly has no g_idx permutation (otherwise quality/correctness breaks)
3. Run on GPU/CPU where desc_act is supported

Example fix

# quantization_config.json before
{"desc_act": true}
# after
{"desc_act": false}  # requires a checkpoint quantized without act-order

cfg = json.load(open(f"{model_path}/quantization_config.json"))
if cfg.get("quant_method") == "gptq" and cfg.get("desc_act", False):
    raise SystemExit("NPU GPTQ requires desc_act=false; requantize without act-order")

def npu_gptq_ok(cfg: dict) -> bool:
    return cfg.get("quant_method") != "gptq" or cfg.get("desc_act", False) is False

try:
    load_model(model_path)
except ValueError as e:
    if "desc_act" in str(e) and "npu" in str(e):
        raise SystemExit("Use a non-act-order GPTQ checkpoint for Ascend NPU")
    raise

When it happens

Trigger: Loading a GPTQ checkpoint whose quantization_config sets desc_act: true (act-order) on an npu backend; create_weights of the Ascend scheme raises before delegating to the base scheme.

Common situations: Most modern GPTQ exports (GPTQModel/auto-gptq) enable desc_act by default; deploying one of those checkpoints on Huawei Ascend NPU hits this at model load.

Related errors

• {name} is required for NPU packed attention
• {name} must be a 1D int32 or int64 tensor
• {name} and its host copy must have the same length
• {name} must start with 0 and contain at least one sequence
• {name} must end at the packed token count {total_tokens}, go

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/79a403568f8c9808. Report an issue: GitHub.