{"record":{"id":"5c84e15f4be2c1a6","repo":"sgl-project/sglang","slug":"quanto-layer-prefix-r-has-incompatible-scale-sc","errorCode":null,"errorMessage":"Quanto layer {prefix!r} has incompatible scale {scale_slice.get_dtype()} {scale_shape}","messagePattern":"Quanto layer (.+?) has incompatible scale (.+?) (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py","lineNumber":168,"sourceCode":"            if f\"{prefix}.weight\" in checkpoint_keys:\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} contains both packed and dense weights\"\n                )\n\n            data_slice = checkpoint.get_slice(names[\"data\"])\n            scale_slice = checkpoint.get_slice(names[\"scale\"])\n            data_shape = tuple(data_slice.get_shape())\n            scale_shape = tuple(scale_slice.get_shape())\n            if data_slice.get_dtype() != \"I8\" or len(data_shape) != 2:\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} needs a 2D I8 weight, got \"\n                    f\"{data_slice.get_dtype()} {data_shape}\"\n                )\n            if scale_slice.get_dtype() not in _FLOAT_DTYPES or scale_shape != (\n                data_shape[0],\n                1,\n            ):\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} has incompatible scale \"\n                    f\"{scale_slice.get_dtype()} {scale_shape}\"\n                )\n            for scale_name in (names[\"input\"], names[\"output\"]):\n                scale = checkpoint.get_slice(scale_name)\n                if (\n                    scale.get_dtype() not in _FLOAT_DTYPES\n                    or tuple(scale.get_shape()) != ()\n                ):\n                    raise ValueError(\n                        f\"Quanto auxiliary scale {scale_name!r} must be a float scalar\"\n                    )\n\n            mapped_prefix = (\n                param_name_mapper(prefix) if param_name_mapper is not None else prefix\n            )\n            if mapped_prefix in mapped_prefixes:\n                raise ValueError(","sourceCodeStart":150,"sourceCodeEnd":186,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py#L150-L186","documentation":"The per-row scale tensor '<prefix>.weight._scale' must be a float dtype and have shape (out_features, 1) matching the 2D _data's first dimension. This error fires when the scale dtype isn't float or its shape doesn't equal (data_shape[0], 1).","triggerScenarios":"A _scale tensor stored as int/half-unsupported dtype, or with shape like (), (1,), (out, out), (1, in) — anything other than (rows, 1) — for a prefix in the quantization_map.","commonSituations":"Squeezing the scale to a scalar for per-tensor quantization, transposed weight export so the scale axis doesn't match, or a different quanto version writing scales with a different layout.","solutions":["Reshape/save _scale as float32/bfloat16/float16 with shape (out_features, 1)","Ensure the weight was quantized per-row (per output channel) not per-tensor; re-run quanto weight-only int8 quantization if unsure"],"exampleFix":"# before: scale saved as scalar (per-tensor)\n# 'enc.0.weight._scale': shape ()\n# after: per-row scale\nscale = scale.reshape(data_shape[0], 1)  # float dtype, e.g. float32","handlingStrategy":"validation","validationCode":"for p in quantization_map:\n    d = ckpt.get_slice(f'{p}.weight._data'); s = ckpt.get_slice(f'{p}.weight._scale')\n    if s.get_dtype() not in ('F32','BF16','F16') or tuple(s.get_shape()) != (d.get_shape()[0], 1):\n        raise SystemExit(f'{p}: bad scale {s.get_dtype()} {s.get_shape()}')","typeGuard":null,"tryCatchPattern":"try:\n    cfg = inspect_quanto_int8_checkpoint(ckpt, mapper)\nexcept ValueError as e:\n    if 'incompatible scale' in str(e):\n        raise SystemExit('Reshape _scale to (out_features, 1) float')\n    raise","preventionTips":["Quantize per-output-channel so scales come out as (out,1)","Don't squeeze or transpose scale tensors during export"],"tags":["quantization","quanto","scale","shape-validation"],"backgroundTag":"invalid-tensor-dtype-shape","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}