{"record":{"id":"85a8240a7e88a92e","repo":"sgl-project/sglang","slug":"comfy-nvfp4-layer-prefix-r-needs-u8-packed-weigh","errorCode":null,"errorMessage":"Comfy NVFP4 layer {prefix!r} needs U8 packed weights and FP8 block scales, got {weight_dtype} and {scale_dtype}","messagePattern":"Comfy NVFP4 layer (.+?) needs U8 packed weights and FP8 block scales, got (.+?) and (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":255,"sourceCode":"            convrot_group_size = int(marker.get(\"convrot_groupsize\", 256))\n            if convrot_group_size not in (16, 64, 256):\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has unsupported \"\n                    f\"convrot_groupsize={convrot_group_size}\"\n                )\n            if logical_input_size % 64 or logical_input_size % convrot_group_size:\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has input size \"\n                    f\"{logical_input_size}, incompatible with quant_group_size=64 \"\n                    f\"and convrot_groupsize={convrot_group_size}\"\n                )\n            continue\n        if marker_format == \"nvfp4\":\n            weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_scale\"]\n            scale_2_dtype, scale_2_shape = checkpoint_meta[f\"{prefix}.weight_scale_2\"]\n            if weight_dtype != \"U8\" or scale_dtype != \"F8_E4M3\":\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} needs U8 packed weights and \"\n                    f\"FP8 block scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if scale_2_dtype != \"F32\" or scale_2_shape not in ((), (1,)):\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} needs a scalar F32 \"\n                    f\"weight_scale_2, got {scale_2_dtype}{scale_2_shape}\"\n                )\n            if len(weight_shape) != 2:\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} needs a 2D packed weight, \"\n                    f\"got {weight_shape}\"\n                )\n            logical_input_size = weight_shape[1] * 2\n            expected_scale_shape = (weight_shape[0], logical_input_size // 16)\n            if logical_input_size % 16 or scale_shape != expected_scale_shape:\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} has incompatible weight/scale \"","sourceCodeStart":237,"sourceCodeEnd":273,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L237-L273","documentation":"For `nvfp4` format layers, the checker requires the packed weight tensor dtype to be U8 (two 4-bit values per byte) and the block-scale tensor dtype to be F8_E4M3 (FP8). This matches the NVFP4 kernel contract: UINT4x2 weights with FP8 E4M3 16-element block scales plus a global F32 scale. Any other dtype pair means the checkpoint wasn't produced by a compliant NVFP4 quantizer.","triggerScenarios":"`inspect_comfy_quant_markers` on a checkpoint where a marker has format 'nvfp4' but `{prefix}.weight` is not dtype U8 or `{prefix}.weight_scale` is not F8_E4M3 — e.g. weights saved unpacked as I8/INT4 tensors, or scales stored as F16/BF16/F32.","commonSituations":"Mixing checkpoints quantized with different NVFP4 export recipes; an older/newer exporter that saved FP8 scales as BF16; converting a checkpoint through a tool that re-casts dtypes; mislabeling a layer's format marker as nvfp4 when it is really w4a8 or int8.","solutions":["Inspect the safetensors header for the failing prefix and confirm actual dtypes of weight and weight_scale","Re-export the model with a quantizer that packs NVFP4 weights as U8 and block scales as F8_E4M3 (e.g. the matching Comfy/tensorrt export recipe)","If the format marker is wrong (the layer was quantized as another format), fix the marker's 'format' field","Upgrade SGLang if a newer build accepts the dtype combination you have"],"exampleFix":"# before: weight dtype I8, scale dtype BF16\n# after (re-export): weight dtype U8 (packed fp4x2), weight_scale dtype F8_E4M3","handlingStrategy":"validation","validationCode":"wd, _ = checkpoint_meta[f\"{prefix}.weight\"]\nsd, _ = checkpoint_meta[f\"{prefix}.weight_scale\"]\nif marker.get(\"format\") == \"nvfp4\":\n    assert wd == \"U8\" and sd == \"F8_E4M3\", f\"{prefix}: got {wd}/{sd}\"","typeGuard":"def is_valid_nvfp4_dtypes(meta: dict, prefix: str) -> bool:\n    return (\n        meta[f\"{prefix}.weight\"][0] == \"U8\"\n        and meta[f\"{prefix}.weight_scale\"][0] == \"F8_E4M3\"\n    )","tryCatchPattern":null,"preventionTips":["Verify safetensors header dtypes immediately after export","Keep the NVFP4 export recipe pinned to the version matching your SGLang build","Never convert NVFP4 checkpoints through generic dtype-casting tools"],"tags":["quantization","nvfp4","fp8","dtype-mismatch","checkpoint-validation"],"backgroundTag":"quantization-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}