{"record":{"id":"e502ab400cf92305","repo":"sgl-project/sglang","slug":"comfy-nvfp4-layer-prefix-r-has-incompatible-weig","errorCode":null,"errorMessage":"Comfy NVFP4 layer {prefix!r} has incompatible weight/scale shapes: {weight_shape} and {scale_shape}","messagePattern":"Comfy NVFP4 layer (.+?) has incompatible weight/scale shapes: (.+?) and (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":272,"sourceCode":"            if weight_dtype != \"U8\" or scale_dtype != \"F8_E4M3\":\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} needs U8 packed weights and \"\n                    f\"FP8 block scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if scale_2_dtype != \"F32\" or scale_2_shape not in ((), (1,)):\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} needs a scalar F32 \"\n                    f\"weight_scale_2, got {scale_2_dtype}{scale_2_shape}\"\n                )\n            if len(weight_shape) != 2:\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} needs a 2D packed weight, \"\n                    f\"got {weight_shape}\"\n                )\n            logical_input_size = weight_shape[1] * 2\n            expected_scale_shape = (weight_shape[0], logical_input_size // 16)\n            if logical_input_size % 16 or scale_shape != expected_scale_shape:\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} has incompatible weight/scale \"\n                    f\"shapes: {weight_shape} and {scale_shape}\"\n                )\n            pre_quant_scale_key = f\"{prefix}.pre_quant_scale\"\n            marker[\"_has_pre_quant_scale\"] = pre_quant_scale_key in checkpoint_meta\n            if marker[\"_has_pre_quant_scale\"]:\n                pre_scale_dtype, pre_scale_shape = checkpoint_meta[pre_quant_scale_key]\n                if pre_scale_dtype not in (\"BF16\", \"F16\", \"F32\") or (\n                    pre_scale_shape != (logical_input_size,)\n                ):\n                    raise ValueError(\n                        f\"Comfy NVFP4 layer {prefix!r} has an incompatible \"\n                        f\"pre_quant_scale: {pre_scale_dtype}{pre_scale_shape}\"\n                    )\n            continue\n        if marker_format != \"int8_tensorwise\":\n            continue\n        weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]","sourceCodeStart":254,"sourceCodeEnd":290,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L254-L290","documentation":"For nvfp4 layers, after unpacking (logical_input_size = weight_shape[1]*2) the checker requires logical_input_size divisible by 16 (the FP4 block size) and weight_scale shape exactly (weight_shape[0], logical_input_size//16) — one FP8 scale per 16 quantized values. Any deviation means scale and weight layouts disagree and dequantization would be incorrect.","triggerScenarios":"`inspect_comfy_quant_markers` on an nvfp4 layer where the scale tensor's second dim doesn't equal in_features/16, the first dim doesn't equal out_features, or in_features isn't a multiple of 16. E.g. scale saved with block size 32, or a transposed weight.","commonSituations":"Exporter using a different block size than 16; transposed weight/scale conventions between exporter versions; checkpoint resized or pruned after quantization; partial-tensor merging.","solutions":["Print weight_shape and scale_shape for the failing prefix and compute expected (shape[0], shape[1]*2//16)","Re-export with block size 16 scales matching the packed weight layout","If the weight is transposed relative to the scale, fix the export orientation","If in_features isn't a multiple of 16, exclude that layer from nvfp4 quantization"],"exampleFix":"# before: weight (3072, 2048), scale (3072, 2048)  # block size wrong\n# after: scale (3072, 4096//16) = (3072, 256)","handlingStrategy":"validation","validationCode":"_, w_shape = checkpoint_meta[f\"{prefix}.weight\"]\n_, s_shape = checkpoint_meta[f\"{prefix}.weight_scale\"]\nlogical_in = w_shape[1] * 2\nexpected = (w_shape[0], logical_in // 16)\nassert logical_in % 16 == 0 and s_shape == expected, f\"expected scale {expected}, got {s_shape}\"","typeGuard":"def nvfp4_scale_shape_matches(w_shape, s_shape) -> bool:\n    logical_in = w_shape[1] * 2\n    return logical_in % 16 == 0 and s_shape == (w_shape[0], logical_in // 16)","tryCatchPattern":null,"preventionTips":["Cross-check scale shape against (out, in/16) in the export script's self-test","Never resize/prune quantized tensors without recomputing scales","Pin the exporter's block size to 16 for NVFP4"],"tags":["quantization","nvfp4","shape-mismatch","block-scale","checkpoint-validation"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}