sgl-project/sglang · critical · ValueError

Serialized W4A8 layer {prefix!r} must set convrot=true

Error message

Serialized W4A8 layer {prefix!r} must set convrot=true

What it means

Every asym_w4a8_int8 marker must explicitly set convrot=true; the W4A8 kernel assumes ConvRot-transformed weights and cannot run otherwise. Missing/false convrot aborts config construction.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py:62

                    f"{capability.to_int() / 10:.1f}"
                )
        self.layer_markers = layer_markers
        self.checkpoint_uses_native_qkv_layout = True
        self.selected: list[str] = []

        for prefix, marker in layer_markers.items():
            marker_format = marker.get("format")
            if marker_format == "int8_tensorwise" and marker.get(
                "_is_tensorwise_scalar"
            ):
                continue
            if marker_format != "asym_w4a8_int8":
                raise ValueError(
                    f"Unsupported Comfy W4A8 format for {prefix!r}: "
                    f"{marker_format!r}"
                )
            if marker.get("convrot") is not True:
                raise ValueError(
                    f"Serialized W4A8 layer {prefix!r} must set convrot=true"
                )

    @classmethod
    def get_name(cls) -> str:
        return "kitchen_w4a8"

    @classmethod
    def get_supported_act_dtypes(cls) -> list[torch.dtype]:
        return [torch.bfloat16, torch.float16]

    @classmethod
    def get_min_capability(cls) -> int:
        return 80

    @classmethod
    def get_config_filenames(cls) -> list[str]:
        return []

View on GitHub (pinned to 0132848349)

Solutions

1. Add "convrot": true to every asym_w4a8_int8 marker
2. Re-export with a current exporter

Example fix

// before
{"format": "asym_w4a8_int8"}
// after
{"format": "asym_w4a8_int8", "convrot": true}

for prefix, m in layer_markers.items():
    if m.get("format") == "asym_w4a8_int8":
        assert m.get("convrot") is True, prefix

def w4a8_marker_complete(m: dict) -> bool:
    return m.get("format") == "asym_w4a8_int8" and m.get("convrot") is True

When it happens

Trigger: A W4A8 marker where 'convrot' is absent, False, or not exactly True.

Common situations: Older exporter versions omitting the flag; manually constructed marker dicts.

Related errors

• Serialized kitchen_int8 layer {prefix!r} must set convrot=tr
• Serialized kitchen_int8 layer {prefix!r} must declare convro
• Unsupported Comfy W4A8 format for {prefix!r}: {marker_format
• Unsupported quantized embedding marker for {prefix!r}: {mark
• SGLang diffusion currently supports AutoRound auto_gptq chec

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/91001c3f91e58aee. Report an issue: GitHub.