{"record":{"id":"b9825390f5c3ae68","repo":"sgl-project/sglang","slug":"comfy-int8-layer-prefix-r-has-incompatible-weigh","errorCode":null,"errorMessage":"Comfy INT8 layer {prefix!r} has incompatible weight/scale shapes: {weight_shape} and {scale_shape}","messagePattern":"Comfy INT8 layer (.+?) has incompatible weight/scale shapes: (.+?) and (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":306,"sourceCode":"        if marker_format != \"int8_tensorwise\":\n            continue\n        weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n        scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_scale\"]\n        if weight_dtype == \"I8\" and scale_dtype == \"F32\" and scale_shape == ():\n            if len(weight_shape) != 2:\n                raise ValueError(\n                    f\"Comfy tensorwise INT8 layer {prefix!r} needs a 2D weight, \"\n                    f\"got {weight_shape}\"\n                )\n            marker[\"_is_tensorwise_scalar\"] = True\n            continue\n        if weight_dtype != \"I8\" or scale_dtype != \"F32\":\n            raise ValueError(\n                f\"Comfy INT8 layer {prefix!r} needs I8 weights and F32 scales, \"\n                f\"got {weight_dtype} and {scale_dtype}\"\n            )\n        if len(weight_shape) != 2 or scale_shape != (weight_shape[0], 1):\n            raise ValueError(\n                f\"Comfy INT8 layer {prefix!r} has incompatible weight/scale \"\n                f\"shapes: {weight_shape} and {scale_shape}\"\n            )\n        marker[\"_is_rowwise\"] = True\n\n    mapped_markers: dict[str, dict[str, Any]] = {}\n    for prefix, marker in raw_markers.items():\n        mapped_prefix = param_name_mapper(prefix) if param_name_mapper else prefix\n        if mapped_prefix in mapped_markers:\n            raise ValueError(\n                f\"Comfy markers collide after parameter mapping at {mapped_prefix!r}\"\n            )\n        mapped_markers[mapped_prefix] = marker\n    return mapped_markers\n\n\ndef resolve_comfy_checkpoint_quantization(\n    layer_markers: dict[str, dict[str, Any]],","sourceCodeStart":288,"sourceCodeEnd":324,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L288-L324","documentation":"For rowwise int8 layers (I8 weight, F32 non-scalar scale), the checker requires a 2D weight and scale shape exactly (out_features, 1) — one scale per output row. Any other combination (non-2D weight, per-column scale, scalar scale with wrong path, mismatched first dim) means dequantization would misalign.","triggerScenarios":"`inspect_comfy_quant_markers` on an int8_tensorwise marker where weight is I8, scale is F32 non-scalar, but scale_shape != (weight_shape[0], 1) or the weight isn't 2D — e.g. scale saved as (1, in_features) (columnwise) or (out,) flattened.","commonSituations":"Exporter emitting columnwise instead of rowwise scales; transposed weights between exporter versions; conv weights left 4D; scale flattened to 1D during conversion.","solutions":["Print weight_shape and scale_shape for the failing prefix; expected scale is (weight_shape[0], 1)","Re-export with per-output-row scales, or transpose/reformat the scale tensor to match","If the weight isn't 2D, flatten conv kernels or exclude the layer from int8 quantization","Ensure the exporter's row/column convention matches SGLang's expected rowwise layout"],"exampleFix":"# before: weight (3072, 4096), scale (1, 3072)  # columnwise\n# after: scale (3072, 1)  # one F32 scale per output row","handlingStrategy":"validation","validationCode":"_, w_shape = checkpoint_meta[f\"{prefix}.weight\"]\n_, s_shape = checkpoint_meta[f\"{prefix}.weight_scale\"]\nassert len(w_shape) == 2 and s_shape == (w_shape[0], 1), (\n    f\"rowwise INT8 scale must be {(w_shape[0], 1)}, got {s_shape}\"\n)","typeGuard":"def rowwise_int8_shapes_ok(w_shape, s_shape) -> bool:\n    return len(w_shape) == 2 and s_shape == (w_shape[0], 1)","tryCatchPattern":null,"preventionTips":["Emit one F32 scale per output row (out, 1), not columnwise or flattened","Verify weight orientation (out, in) matches the exporter's convention in every release","Automated post-export shape audit: assert scale.shape == (weight.shape[0], 1) for all rowwise int8 layers"],"tags":["quantization","int8","rowwise","shape-mismatch","checkpoint-validation"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}