sgl-project/sglang · error · ValueError

Unsupported quantized linear marker for {prefix!r}

Error message

Unsupported quantized linear marker for {prefix!r}

What it means

Thrown by KitchenW4A8Config.get_quant_method when a quantization marker exists on a linear layer but its 'format' field is not 'asym_w4a8_int8'. The config only supports asymmetric W4A8 int8 weights, so any other serialized format is rejected at layer dispatch time.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py:109

    ) -> QuantizeMethodBase | None:
        marker = self.layer_markers.get(prefix)
        if isinstance(layer, VocabParallelEmbedding):
            if marker is None:
                return None
            if marker.get("format") != "int8_tensorwise" or not marker.get(
                "_is_tensorwise_scalar"
            ):
                raise ValueError(
                    f"Unsupported quantized embedding marker for {prefix!r}: {marker}"
                )
            self.selected.append(prefix)
            return KitchenInt8EmbeddingMethod()
        if not isinstance(layer, LinearBase):
            return None
        if marker is None:
            return UnquantizedLinearMethod()
        if marker.get("format") != "asym_w4a8_int8":
            raise ValueError(f"Unsupported quantized linear marker for {prefix!r}")

        group_size = int(marker.get("group_size", 16))
        convrot_group_size = int(marker.get("convrot_groupsize", 256))
        if not self._supports_input_size(
            layer.input_size, group_size, convrot_group_size
        ):
            raise ValueError(
                f"Serialized W4A8 layer {prefix!r} has input size "
                f"{layer.input_size}, incompatible with group_size={group_size} "
                f"and convrot_groupsize={convrot_group_size}"
            )
        self.selected.append(prefix)
        return KitchenW4A8LinearMethod(
            group_size=group_size,
            convrot_group_size=convrot_group_size,
            has_codebook=bool(marker.get("_has_codebook")),
            has_correction=bool(marker.get("_has_correction")),
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect the checkpoint's marker metadata to confirm the actual format string
2. Re-quantize/export the checkpoint with format 'asym_w4a8_int8'
3. Extend get_quant_method to handle the new format and raise on a narrower set
4. Fall back to UnquantizedLinearMethod only if unquantized weights are available

Example fix

# before
marker = {"format": "sym_w4a8_int8", "group_size": 16}
method = config.get_quant_method(layer, prefix)  # ValueError

# after
marker = {"format": "asym_w4a8_int8", "group_size": 16}
method = config.get_quant_method(layer, prefix)

marker = getattr(layer, "marker", None)
if marker is not None and marker.get("format") != "asym_w4a8_int8":
    raise SystemExit(f"checkpoint uses format {marker.get('format')!r}; re-export as asym_w4a8_int8")

def is_asym_w4a8_marker(marker: object) -> bool:
    return (
        isinstance(marker, dict)
        and marker.get("format") == "asym_w4a8_int8"
        and isinstance(marker.get("group_size", 16), int)
    )

When it happens

Trigger: Calling get_quant_method(layer, prefix) where the layer's marker dict has marker['format'] != 'asym_w4a8_int8' (e.g. 'sym_w4a8', 'w8a8', or a typo), typically while loading a Kitchen-quantized checkpoint whose markers were written by a different quantization recipe.

Common situations: Checkpoint quantized with a newer/older kitchen format string; hand-edited marker dicts; mixing checkpoints serialized with symmetric vs asymmetric W4A8 schemes.

Related errors

• Unsupported Comfy INT8 format for {prefix!r}: {marker.get('f
• Unsupported Comfy W4A4 format for {prefix!r}: {marker_format
• SGLang diffusion currently supports AutoRound auto_gptq chec
• AutoRound fused module {target!r} has inconsistent shard con
• Parameter {param_name} not found in the model.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8f44252250ccc479. Report an issue: GitHub.