sgl-project/sglang · critical · ValueError

Serialized kitchen_int8 layer {prefix!r} must set convrot=tr

Error message

Serialized kitchen_int8 layer {prefix!r} must set convrot=true

What it means

Every serialized kitchen_int8 marker must explicitly set convrot=true. ConvRot (convolution-style weight rotation) is mandatory for the kitchen_int8 kernel path; a missing or false value means the checkpoint was produced by an incompatible or older exporter.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_int8_config.py:54

                f"kitchen_int8 group_size must be one of {_SUPPORTED_GROUP_SIZES}, "
                f"got {group_size}"
            )
        self.group_size = group_size
        self.ignored_layers = ignored_layers or []
        self.packed_modules_mapping = packed_modules_mapping or {}
        self.layer_markers = layer_markers
        self.is_checkpoint_int8_serialized = layer_markers is not None
        self.checkpoint_uses_native_qkv_layout = self.is_checkpoint_int8_serialized
        self._serialized_group_sizes: dict[str, int] = {}
        if layer_markers is not None:
            for prefix, marker in layer_markers.items():
                if marker.get("format") != "int8_tensorwise":
                    raise ValueError(
                        f"Unsupported Comfy INT8 format for {prefix!r}: "
                        f"{marker.get('format')!r}"
                    )
                if marker.get("convrot") is not True:
                    raise ValueError(
                        f"Serialized kitchen_int8 layer {prefix!r} must set "
                        "convrot=true"
                    )
                marker_group_size = marker.get("convrot_groupsize")
                if marker_group_size not in _SUPPORTED_GROUP_SIZES:
                    raise ValueError(
                        f"Serialized kitchen_int8 layer {prefix!r} must declare "
                        f"convrot_groupsize in {_SUPPORTED_GROUP_SIZES}, got "
                        f"{marker_group_size!r}"
                    )
                self._serialized_group_sizes[prefix] = marker_group_size
        # Which layers actually got quantized is worth stating plainly in the
        # log: a silent fallback to BF16 looks exactly like a slow kernel.
        self.selected: list[str] = []
        self.skipped: list[str] = []
        self._processed = 0
        self._quantized_bytes = 0

View on GitHub (pinned to 0132848349)

Solutions

1. Re-export with a current exporter that sets convrot: true
2. Manually add "convrot": true to each marker if you know the weights were ConvRot-quantized

Example fix

// before
{"format": "int8_tensorwise", "convrot_groupsize": 128}
// after
{"format": "int8_tensorwise", "convrot": true, "convrot_groupsize": 128}

for prefix, m in layer_markers.items():
    assert m.get("convrot") is True, f"{prefix} missing convrot=true"

def has_convrot(m: dict) -> bool:
    return m.get("convrot") is True

When it happens

Trigger: Passing a layer_markers entry with 'convrot' absent, False, or a truthy non-True value while format is 'int8_tensorwise'.

Common situations: Hand-writing marker dicts and forgetting the convrot key; using an older Comfy exporter that did not emit convrot flags.

Related errors

• Serialized kitchen_int8 layer {prefix!r} must declare convro
• Serialized W4A8 layer {prefix!r} must set convrot=true
• SGLang diffusion currently supports AutoRound auto_gptq chec
• AutoRound fused module {target!r} has inconsistent shard con
• Parameter {param_name} not found in the model.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9ad8c1a9a4264f96. Report an issue: GitHub.