sgl-project/sglang · critical · ValueError

Unsupported Comfy W4A8 format for {prefix!r}: {marker_format

Error message

Unsupported Comfy W4A8 format for {prefix!r}: {marker_format!r}

What it means

KitchenW4A8Config.__init__ accepts only markers of format 'asym_w4a8_int8' (plus int8_tensorwise with _is_tensorwise_scalar). Any other format string in layer_markers aborts construction.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py:57

                and capability.to_int() < self.get_min_capability()
            ):
                raise ValueError(
                    "Serialized W4A8 checkpoints require CUDA compute capability "
                    f">= {self.get_min_capability() / 10:.1f}; got "
                    f"{capability.to_int() / 10:.1f}"
                )
        self.layer_markers = layer_markers
        self.checkpoint_uses_native_qkv_layout = True
        self.selected: list[str] = []

        for prefix, marker in layer_markers.items():
            marker_format = marker.get("format")
            if marker_format == "int8_tensorwise" and marker.get(
                "_is_tensorwise_scalar"
            ):
                continue
            if marker_format != "asym_w4a8_int8":
                raise ValueError(
                    f"Unsupported Comfy W4A8 format for {prefix!r}: "
                    f"{marker_format!r}"
                )
            if marker.get("convrot") is not True:
                raise ValueError(
                    f"Serialized W4A8 layer {prefix!r} must set convrot=true"
                )

    @classmethod
    def get_name(cls) -> str:
        return "kitchen_w4a8"

    @classmethod
    def get_supported_act_dtypes(cls) -> list[torch.dtype]:
        return [torch.bfloat16, torch.float16]

    @classmethod
    def get_min_capability(cls) -> int:

View on GitHub (pinned to 0132848349)

Solutions

1. Re-export with asym_w4a8_int8 markers
2. Load under the config matching the actual formats (kitchen_w4a4 for convrot_w4a4, kitchen_int8 for int8_tensorwise rows)

Example fix

// before
{"format": "convrot_w4a4"}
// after
{"format": "asym_w4a8_int8", "convrot": true}

for prefix, m in layer_markers.items():
    fmt = m.get("format")
    ok = fmt == "asym_w4a8_int8" or (fmt == "int8_tensorwise" and m.get("_is_tensorwise_scalar"))
    assert ok, (prefix, fmt)

def is_w4a8_loadable_marker(m: dict) -> bool:
    fmt = m.get("format")
    return fmt == "asym_w4a8_int8" or (fmt == "int8_tensorwise" and bool(m.get("_is_tensorwise_scalar")))

When it happens

Trigger: A marker with format 'nvfp4', 'convrot_w4a4', or plain 'int8_tensorwise' without _is_tensorwise_scalar while under kitchen_w4a8.

Common situations: Wrong quant config selected for a mixed checkpoint; exporter emitting a different format name.

Related errors

• Serialized W4A8 layer {prefix!r} must set convrot=true
• Unsupported quantized embedding marker for {prefix!r}: {mark
• Comfy layer {prefix!r} is missing checkpoint tensors: {sorte
• Comfy W4A8 layer {prefix!r} has invalid group_size={group_si
• SGLang diffusion currently supports AutoRound auto_gptq chec

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7d00930c836ff9d7. Report an issue: GitHub.