{"record":{"id":"592ee17ba9d2d430","repo":"sgl-project/sglang","slug":"comfy-nvfp4-layer-prefix-r-needs-a-2d-packed-wei","errorCode":null,"errorMessage":"Comfy NVFP4 layer {prefix!r} needs a 2D packed weight, got {weight_shape}","messagePattern":"Comfy NVFP4 layer (.+?) needs a 2D packed weight, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":265,"sourceCode":"                    f\"and convrot_groupsize={convrot_group_size}\"\n                )\n            continue\n        if marker_format == \"nvfp4\":\n            weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_scale\"]\n            scale_2_dtype, scale_2_shape = checkpoint_meta[f\"{prefix}.weight_scale_2\"]\n            if weight_dtype != \"U8\" or scale_dtype != \"F8_E4M3\":\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} needs U8 packed weights and \"\n                    f\"FP8 block scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if scale_2_dtype != \"F32\" or scale_2_shape not in ((), (1,)):\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} needs a scalar F32 \"\n                    f\"weight_scale_2, got {scale_2_dtype}{scale_2_shape}\"\n                )\n            if len(weight_shape) != 2:\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} needs a 2D packed weight, \"\n                    f\"got {weight_shape}\"\n                )\n            logical_input_size = weight_shape[1] * 2\n            expected_scale_shape = (weight_shape[0], logical_input_size // 16)\n            if logical_input_size % 16 or scale_shape != expected_scale_shape:\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} has incompatible weight/scale \"\n                    f\"shapes: {weight_shape} and {scale_shape}\"\n                )\n            pre_quant_scale_key = f\"{prefix}.pre_quant_scale\"\n            marker[\"_has_pre_quant_scale\"] = pre_quant_scale_key in checkpoint_meta\n            if marker[\"_has_pre_quant_scale\"]:\n                pre_scale_dtype, pre_scale_shape = checkpoint_meta[pre_quant_scale_key]\n                if pre_scale_dtype not in (\"BF16\", \"F16\", \"F32\") or (\n                    pre_scale_shape != (logical_input_size,)\n                ):\n                    raise ValueError(","sourceCodeStart":247,"sourceCodeEnd":283,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L247-L283","documentation":"The NVFP4 code path assumes a 2D packed weight matrix (out_features, in_features/2 packed bytes). If `{prefix}.weight` has any other rank — e.g. a 4D conv kernel or a 1D vector — the subsequent shape arithmetic (logical_input_size = weight_shape[1]*2, expected scale shape) would be meaningless, so the checker rejects it up front.","triggerScenarios":"`inspect_comfy_quant_markers` on an nvfp4 marker whose weight tensor is not 2D — typically a conv layer exported without being reshaped into a 2D matrix, or a mistakenly tagged layer.","commonSituations":"Quantizing convolutional/patch-embed layers as nvfp4 without flattening spatial dims to a matrix; exporter bugs that leave conv weights 4D; format marker applied to the wrong module prefix.","solutions":["Inspect weight_shape for the failing prefix to confirm its rank","Re-export with the conv weight flattened to (out_channels, in_channels*kh*kw/2) if the kernel supports matrix form","If the layer genuinely can't be 2D, exclude it from nvfp4 quantization (leave it higher precision) in the export recipe","Verify the marker's 'format' field matches the actual tensor layout"],"exampleFix":"# before: weight shape (256, 128, 3, 3)\n# after (flattened, packed): weight shape (256, 2304)  # 4608/2 bytes","handlingStrategy":"validation","validationCode":"_, w_shape = checkpoint_meta[f\"{prefix}.weight\"]\nif marker.get(\"format\") == \"nvfp4\":\n    assert len(w_shape) == 2, f\"{prefix}: expected 2D packed weight, got {w_shape}\"","typeGuard":"def is_2d_weight(meta: dict, prefix: str) -> bool:\n    return len(meta[f\"{prefix}.weight\"][1]) == 2","tryCatchPattern":null,"preventionTips":["Flatten conv weights to 2D in the export pipeline or exclude convs from nvfp4","Add a rank assertion in your export script for every quantized weight","Tag only matrix-multiply layers with the nvfp4 format marker"],"tags":["quantization","nvfp4","rank-mismatch","conv-weights","checkpoint-validation"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}