sgl-project/sglang · error · ValueError

moe_wna16 only support gptq and awq.

Error message

moe_wna16 only support gptq and awq.

What it means

The MoeWNA16Config constructor only accepts quant_method values 'gptq' and 'awq' — these are the only backends with kernels wired into the moe_wna16 scheme. Any other method string in the MoE quant config raises this ValueError at config construction time.

Source

Thrown at python/sglang/srt/layers/quantization/moe_wna16.py:110

        if self.linear_quant_method == "gptq":
            self.use_marlin = GPTQMarlinConfig.is_gptq_marlin_compatible(full_config)
        elif self.linear_quant_method == "awq":
            capability_tuple = get_device_capability()
            device_capability = (
                -1
                if capability_tuple is None
                else capability_tuple[0] * 10 + capability_tuple[1]
            )
            awq_min_capability = AWQConfig.get_min_capability()
            if device_capability < awq_min_capability:
                raise ValueError(
                    "The quantization method moe_wna16 + awq is not supported "
                    "for the current GPU. "
                    f"Minimum capability: {awq_min_capability}. "
                    f"Current capability: {device_capability}."
                )
        else:
            raise ValueError("moe_wna16 only support gptq and awq.")

        if modules_to_not_convert is None:
            self.modules_to_not_convert = []
        else:
            self.modules_to_not_convert = modules_to_not_convert

    @classmethod
    def get_name(cls) -> str:
        return "moe_wna16"

    @classmethod
    def get_supported_act_dtypes(cls) -> List[torch.dtype]:
        return [torch.bfloat16, torch.half]

    @classmethod
    def get_min_capability(cls) -> int:
        return 70

View on GitHub (pinned to 0132848349)

Solutions

1. Set quant_method in the MoE quant config to exactly 'gptq' or 'awq'
2. If the checkpoint uses another method, pick the matching SGLang quant scheme (e.g. qqq, w8a8) instead of forcing moe_wna16
3. Update SGLang — newer revisions may have added the method you need

Example fix

// before
{"quant_method": "gptq_v2", "bits": 4, ...}
// after
{"quant_method": "gptq", "bits": 4, ...}

method = quant_cfg.get("quant_method")
if method not in ("gptq", "awq"):
    raise ValueError(f"moe_wna16 needs gptq/awq, got {method!r}")
cfg = MoeWNA16Config(method, bits, group, has_zp, ...)

from typing import Literal
QuantMethod = Literal["gptq", "awq"]

def is_supported_method(m: str) -> TypeGuard[QuantMethod]:
    return m in ("gptq", "awq")

When it happens

Trigger: Constructing MoeWNA16Config directly, or via from_config, where the quant_method resolved from the config dict is something like 'qqq', 'gptq_v2', 'awq_v2', or a typo like 'awq '.

Common situations: quant_config.json of a MoE checkpoint carries an unrecognized quant_method key; hand-edited configs; or a newly published quant variant that moe_wna16 doesn't implement yet.

Related errors

• num_bits must be 4 or 8, got {}
• The hpc_ops MoE runner backend only supports FP8-quantized M
• Currently, only group size 128 and -1 (channelwise) is suppo
• The quantization method moe_wna16 + awq is not supported for
• kitchen_int8 group_size must be one of {_SUPPORTED_GROUP_SIZ

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1bc7cab302840564. Report an issue: GitHub.