sgl-project/sglang · critical · ValueError

Unsupported Comfy INT8 format for {prefix!r}: {marker.get('f

Error message

Unsupported Comfy INT8 format for {prefix!r}: {marker.get('format')!r}

What it means

When KitchenInt8Config is built with layer_markers, every serialized marker must have format 'int8_tensorwise'. A marker with any other format means the checkpoint mixes schemes the kitchen_int8 path cannot dispatch, so construction aborts.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_int8_config.py:49

        layer_markers: dict[str, dict[str, Any]] | None = None,
    ) -> None:
        super().__init__()
        if group_size not in _SUPPORTED_GROUP_SIZES:
            raise ValueError(
                f"kitchen_int8 group_size must be one of {_SUPPORTED_GROUP_SIZES}, "
                f"got {group_size}"
            )
        self.group_size = group_size
        self.ignored_layers = ignored_layers or []
        self.packed_modules_mapping = packed_modules_mapping or {}
        self.layer_markers = layer_markers
        self.is_checkpoint_int8_serialized = layer_markers is not None
        self.checkpoint_uses_native_qkv_layout = self.is_checkpoint_int8_serialized
        self._serialized_group_sizes: dict[str, int] = {}
        if layer_markers is not None:
            for prefix, marker in layer_markers.items():
                if marker.get("format") != "int8_tensorwise":
                    raise ValueError(
                        f"Unsupported Comfy INT8 format for {prefix!r}: "
                        f"{marker.get('format')!r}"
                    )
                if marker.get("convrot") is not True:
                    raise ValueError(
                        f"Serialized kitchen_int8 layer {prefix!r} must set "
                        "convrot=true"
                    )
                marker_group_size = marker.get("convrot_groupsize")
                if marker_group_size not in _SUPPORTED_GROUP_SIZES:
                    raise ValueError(
                        f"Serialized kitchen_int8 layer {prefix!r} must declare "
                        f"convrot_groupsize in {_SUPPORTED_GROUP_SIZES}, got "
                        f"{marker_group_size!r}"
                    )
                self._serialized_group_sizes[prefix] = marker_group_size
        # Which layers actually got quantized is worth stating plainly in the
        # log: a silent fallback to BF16 looks exactly like a slow kernel.

View on GitHub (pinned to 0132848349)

Solutions

1. Re-export the checkpoint with all-int8_tensorwise quantization
2. Split mixed markers and load each subset under its matching quant config (e.g. kitchen_w4a4 handles convrot_w4a4)

Example fix

// before
layer_markers = {"blk.0": {"format": "convrot_w4a4", "convrot": True}}
// after
layer_markers = {"blk.0": {"format": "int8_tensorwise", "convrot": True, "convrot_groupsize": 128}}

for prefix, m in layer_markers.items():
    assert m.get("format") == "int8_tensorwise", (prefix, m.get("format"))

def is_int8_tensorwise_marker(m: dict) -> bool:
    return m.get("format") == "int8_tensorwise"

When it happens

Trigger: Passing layer_markers containing an entry whose 'format' key is not 'int8_tensorwise' (e.g. 'nvfp4' or 'convrot_w4a4').

Common situations: Exporting a Comfy model with mixed per-layer quantization and loading it under kitchen_int8; concatenating marker dicts from different exports.

Related errors

• Unsupported Comfy W4A4 format for {prefix!r}: {marker_format
• Unsupported quantized linear marker for {prefix!r}
• SGLang diffusion currently supports AutoRound auto_gptq chec
• AutoRound fused module {target!r} has inconsistent shard con
• Parameter {param_name} not found in the model.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b42357aa049434ff. Report an issue: GitHub.