sgl-project/sglang · error · ValueError

Unsupported num_bits = {num_bits}. Supported num_bits = {WNA

Error message

Unsupported num_bits = {num_bits}. Supported num_bits = {WNA16_SUPPORTED_TYPES_MAP.keys()}

What it means

The wNa16 Marlin scheme only supports specific bit-widths listed in WNA16_SUPPORTED_TYPES_MAP (typically 4 and 8). num_bits outside that set (e.g. 2, 3, 6) has no kernel dtype mapping and is rejected at init.

Source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_wNa16.py:85

                 strategy: str,
                 num_bits: int,
                 group_size: Optional[int] = None,
                 symmetric: Optional[bool] = True,
                 actorder: Optional[ActivationOrdering] = None):

        self.pack_factor = 32 // num_bits
        self.strategy = strategy
        self.symmetric = symmetric
        self.group_size = -1 if group_size is None else group_size
        self.has_g_idx = actorder == ActivationOrdering.GROUP

        if self.group_size == -1 and self.strategy != "channel":
            raise ValueError("Marlin kernels require group quantization or "
                             "channelwise quantization, but found no group "
                             "size and strategy is not channelwise.")

        if num_bits not in WNA16_SUPPORTED_TYPES_MAP:
            raise ValueError(
                f"Unsupported num_bits = {num_bits}. "
                f"Supported num_bits = {WNA16_SUPPORTED_TYPES_MAP.keys()}")

        self.quant_type = (WNA16_ZP_SUPPORTED_TYPES_MAP[num_bits]
                           if not self.symmetric else
                           WNA16_SUPPORTED_TYPES_MAP[num_bits])

    @classmethod
    def get_min_capability(cls) -> int:
        # ampere and up
        return 80

    def create_weights(self, layer: torch.nn.Module, output_size: int,
                       input_size: int, output_partition_sizes: list[int],
                       input_size_per_partition: int,
                       params_dtype: torch.dtype, weight_loader: Callable,
                       **kwargs):

View on GitHub (pinned to 0132848349)

Solutions

1. Re-quantize to a supported bit width (4 or 8)
2. Check the error's printed supported keys against your config
3. Upgrade SGLang if wider bit support was added upstream

Example fix

// before
"weights": {"num_bits": 3, "strategy": "group", "group_size": 128}
// after
"weights": {"num_bits": 4, "strategy": "group", "group_size": 128}

SUPPORTED = {4, 8}
def num_bits_ok(w):
    return w["num_bits"] in SUPPORTED

When it happens

Trigger: A compressed-tensors checkpoint with weights.num_bits not in the supported set (commonly anything other than 4/8) initializing CompressedTensorsWNA16.

Common situations: Exotic bit-widths from experimental quantizers; AWQ/GPTQ variants with 3-bit weights.

Related errors

• Online quantization {explicit_quantization!r} is not support
• Unsupported Comfy NVFP4 companion format(s): + ", ".join(sor
• Marlin kernels require group quantization or channelwise qua
• For Fused MoE layers, only {CompressionFormat.pack_quantized
• num_bits must be 4 or 8, got {}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b145fb29cf07d908. Report an issue: GitHub.