sgl-project/sglang · error · ValueError

Comfy W4A8 layer {prefix!r} needs an F32[16] codebook

Error message

Comfy W4A8 layer {prefix!r} needs an F32[16] codebook

What it means

An optional per-layer weight codebook ({prefix}.weight_codebook) for a Comfy W4A8 layer must be exactly a float32 tensor of length 16 (the 16 possible 4-bit codes). If present but with a different dtype or shape, this error is raised because codebook lookup would be invalid.

Source

Thrown at python/sglang/multimodal_gen/runtime/utils/quantization_utils.py:208

                )
            logical_input_size = weight_shape[1] * 2
            expected_scale_shape = (weight_shape[0], logical_input_size // group_size)
            if scale_shape != expected_scale_shape or channel_shape != (
                weight_shape[0],
            ):
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} has incompatible weight/scale "
                    f"shapes: {weight_shape}, {scale_shape}, and {channel_shape}"
                )
            codebook_key = f"{prefix}.weight_codebook"
            correction_key = f"{prefix}.weight_correction"
            marker["_has_codebook"] = codebook_key in checkpoint_meta
            marker["_has_correction"] = correction_key in checkpoint_meta
            if marker["_has_codebook"] and checkpoint_meta[codebook_key] != (
                "F32",
                (16,),
            ):
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs an F32[16] codebook"
                )
            expected_correction = (
                logical_input_size // group_size,
                weight_shape[0],
            )
            if marker["_has_correction"] and checkpoint_meta[correction_key] != (
                "F32",
                expected_correction,
            ):
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} has an incompatible correction tensor"
                )
            continue
        if marker_format == "convrot_w4a4":
            weight_dtype, weight_shape = checkpoint_meta[f"{prefix}.weight"]
            scale_dtype, scale_shape = checkpoint_meta[f"{prefix}.weight_scale"]
            if weight_dtype != "I8" or scale_dtype != "F32":

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect {prefix}.weight_codebook's dtype/shape in safetensors metadata; it must be F32 with shape (16,)
2. Re-export with a compatible Comfy W4A8 exporter version
3. If the codebook is from a newer format (e.g. 256-entry 8-bit), remove it or upgrade the runtime to the matching version

Example fix

# before: codebook BF16 (16,) or F32 (256,) -> raises
# after:  codebook F32 (16,)
torch.save_codebook(codebook.float().reshape(16))

cb = meta.get(f"{prefix}.weight_codebook")
if cb is not None and cb != ("F32", (16,)):
    raise ValueError(f"unsupported codebook {cb}")

def has_valid_codebook(meta: dict, prefix: str) -> bool:
    cb = meta.get(f"{prefix}.weight_codebook")
    return cb is None or cb == ("F32", (16,))

When it happens

Trigger: inspect_comfy_quant_markers finds {prefix}.weight_codebook in the checkpoint whose safetensors metadata is not exactly ('F32', (16,)) — e.g. BF16 dtype, length 15/256, or a 2D tensor.

Common situations: Exporter version change that emits a larger codebook (e.g. 8-bit with 256 entries) under the same tensor name, or a dtype-conversion pass that altered the codebook tensor.

Related errors

• Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 group s
• Comfy W4A8 layer {prefix!r} needs F32 channel scales, got {c
• Type must match: {self.a_dtype} != {self.b_dtype}
• int32-packed scale buffers require scale_ue8m0=True
• scale_ue8m0=True requires an int32-packed output_s

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9dbbf988211ff91b. Report an issue: GitHub.