sgl-project/sglang · error · ValueError

Comfy W4A8 layer {prefix!r} needs F32 channel scales, got {c

Error message

Comfy W4A8 layer {prefix!r} needs F32 channel scales, got {channel_dtype}

What it means

Comfy W4A8 layers additionally require a per-channel scale tensor ({prefix}.weight_s_channel) stored as float32. This error fires when that channel-scale tensor exists but its safetensors dtype is not 'F32', which would break the channel-wise dequantization path.

Source

Thrown at python/sglang/multimodal_gen/runtime/utils/quantization_utils.py:182

                "static" if f"{prefix}.input_scale" in checkpoint_meta else "dynamic"
            )
            continue
        if marker_format == "asym_w4a8_int8":
            weight_dtype, weight_shape = checkpoint_meta[f"{prefix}.weight"]
            scale_dtype, scale_shape = checkpoint_meta[f"{prefix}.weight_s_rel"]
            channel_dtype, channel_shape = checkpoint_meta[f"{prefix}.weight_s_channel"]
            group_size = int(marker.get("group_size", 16))
            if group_size < 4:
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} has invalid group_size={group_size}"
                )
            if weight_dtype != "I8" or scale_dtype != "F8_E4M3":
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 "
                    f"group scales, got {weight_dtype} and {scale_dtype}"
                )
            if channel_dtype != "F32":
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs F32 channel scales, "
                    f"got {channel_dtype}"
                )
            if len(weight_shape) != 2:
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs a 2D packed weight, "
                    f"got {weight_shape}"
                )
            logical_input_size = weight_shape[1] * 2
            expected_scale_shape = (weight_shape[0], logical_input_size // group_size)
            if scale_shape != expected_scale_shape or channel_shape != (
                weight_shape[0],
            ):
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} has incompatible weight/scale "
                    f"shapes: {weight_shape}, {scale_shape}, and {channel_shape}"
                )
            codebook_key = f"{prefix}.weight_codebook"

View on GitHub (pinned to 0132848349)

Solutions

1. Check the dtype of {prefix}.weight_s_channel in the safetensors metadata and confirm it is F32
2. Re-export the checkpoint without dtype downcasting of scale tensors
3. If a conversion script halved the scales, restore from the original export

Example fix

# before: weight_s_channel stored as BF16 -> raises
# after:  weight_s_channel stored as F32
save_file({"weight_s_channel": scale_channel.float()}, "model.safetensors")

cdt, _ = meta[f"{prefix}.weight_s_channel"]
if cdt != "F32":
    raise ValueError(f"channel scales must be F32, got {cdt}")

def has_f32_channel_scales(meta: dict, prefix: str) -> bool:
    e = meta.get(f"{prefix}.weight_s_channel")
    return e is not None and e[0] == "F32"

When it happens

Trigger: inspect_comfy_quant_markers inspects {prefix}.weight_s_channel for an asym_w4a8_int8 layer and its metadata dtype is anything other than F32 (e.g. BF16, F16, or F64).

Common situations: A downcasting safetensors re-save step (e.g. convert to bf16 utility applied blindly to all tensors), or an exporter version that emitted channel scales in half precision.

Related errors

• Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 group s
• Comfy W4A8 layer {prefix!r} needs an F32[16] codebook
• Type must match: {self.a_dtype} != {self.b_dtype}
• int32-packed scale buffers require scale_ue8m0=True
• scale_ue8m0=True requires an int32-packed output_s

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/eacc993d765ef28b. Report an issue: GitHub.