{"record":{"id":"9dbbf988211ff91b","repo":"sgl-project/sglang","slug":"comfy-w4a8-layer-prefix-r-needs-an-f32-16-codeb","errorCode":null,"errorMessage":"Comfy W4A8 layer {prefix!r} needs an F32[16] codebook","messagePattern":"Comfy W4A8 layer (.+?) needs an F32\\[16\\] codebook","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":208,"sourceCode":"                )\n            logical_input_size = weight_shape[1] * 2\n            expected_scale_shape = (weight_shape[0], logical_input_size // group_size)\n            if scale_shape != expected_scale_shape or channel_shape != (\n                weight_shape[0],\n            ):\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} has incompatible weight/scale \"\n                    f\"shapes: {weight_shape}, {scale_shape}, and {channel_shape}\"\n                )\n            codebook_key = f\"{prefix}.weight_codebook\"\n            correction_key = f\"{prefix}.weight_correction\"\n            marker[\"_has_codebook\"] = codebook_key in checkpoint_meta\n            marker[\"_has_correction\"] = correction_key in checkpoint_meta\n            if marker[\"_has_codebook\"] and checkpoint_meta[codebook_key] != (\n                \"F32\",\n                (16,),\n            ):\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs an F32[16] codebook\"\n                )\n            expected_correction = (\n                logical_input_size // group_size,\n                weight_shape[0],\n            )\n            if marker[\"_has_correction\"] and checkpoint_meta[correction_key] != (\n                \"F32\",\n                expected_correction,\n            ):\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} has an incompatible correction tensor\"\n                )\n            continue\n        if marker_format == \"convrot_w4a4\":\n            weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_scale\"]\n            if weight_dtype != \"I8\" or scale_dtype != \"F32\":","sourceCodeStart":190,"sourceCodeEnd":226,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L190-L226","documentation":"An optional per-layer weight codebook ({prefix}.weight_codebook) for a Comfy W4A8 layer must be exactly a float32 tensor of length 16 (the 16 possible 4-bit codes). If present but with a different dtype or shape, this error is raised because codebook lookup would be invalid.","triggerScenarios":"inspect_comfy_quant_markers finds {prefix}.weight_codebook in the checkpoint whose safetensors metadata is not exactly ('F32', (16,)) — e.g. BF16 dtype, length 15/256, or a 2D tensor.","commonSituations":"Exporter version change that emits a larger codebook (e.g. 8-bit with 256 entries) under the same tensor name, or a dtype-conversion pass that altered the codebook tensor.","solutions":["Inspect {prefix}.weight_codebook's dtype/shape in safetensors metadata; it must be F32 with shape (16,)","Re-export with a compatible Comfy W4A8 exporter version","If the codebook is from a newer format (e.g. 256-entry 8-bit), remove it or upgrade the runtime to the matching version"],"exampleFix":"# before: codebook BF16 (16,) or F32 (256,) -> raises\n# after:  codebook F32 (16,)\ntorch.save_codebook(codebook.float().reshape(16))","handlingStrategy":"validation","validationCode":"cb = meta.get(f\"{prefix}.weight_codebook\")\nif cb is not None and cb != (\"F32\", (16,)):\n    raise ValueError(f\"unsupported codebook {cb}\")","typeGuard":"def has_valid_codebook(meta: dict, prefix: str) -> bool:\n    cb = meta.get(f\"{prefix}.weight_codebook\")\n    return cb is None or cb == (\"F32\", (16,))","tryCatchPattern":null,"preventionTips":["Pin exporter and runtime versions together","Check optional-tensor metadata before loading, not after"],"tags":["quantization","codebook","dtype-mismatch","w4a8"],"backgroundTag":"quantization-config-invalid","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}