{"record":{"id":"0416e3d22055e05b","repo":"sgl-project/sglang","slug":"quanto-auxiliary-scale-scale-name-r-must-be-a-fl","errorCode":null,"errorMessage":"Quanto auxiliary scale {scale_name!r} must be a float scalar","messagePattern":"Quanto auxiliary scale (.+?) must be a float scalar","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py","lineNumber":178,"sourceCode":"                raise ValueError(\n                    f\"Quanto layer {prefix!r} needs a 2D I8 weight, got \"\n                    f\"{data_slice.get_dtype()} {data_shape}\"\n                )\n            if scale_slice.get_dtype() not in _FLOAT_DTYPES or scale_shape != (\n                data_shape[0],\n                1,\n            ):\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} has incompatible scale \"\n                    f\"{scale_slice.get_dtype()} {scale_shape}\"\n                )\n            for scale_name in (names[\"input\"], names[\"output\"]):\n                scale = checkpoint.get_slice(scale_name)\n                if (\n                    scale.get_dtype() not in _FLOAT_DTYPES\n                    or tuple(scale.get_shape()) != ()\n                ):\n                    raise ValueError(\n                        f\"Quanto auxiliary scale {scale_name!r} must be a float scalar\"\n                    )\n\n            mapped_prefix = (\n                param_name_mapper(prefix) if param_name_mapper is not None else prefix\n            )\n            if mapped_prefix in mapped_prefixes:\n                raise ValueError(\n                    f\"Quanto layers collide after parameter mapping at {mapped_prefix!r}\"\n                )\n            mapped_prefixes.add(mapped_prefix)\n\n    return QuantoInt8Config(mapped_prefixes)\n\n\n__all__ = [\"QuantoInt8Config\", \"inspect_quanto_int8_checkpoint\"]\n","sourceCodeStart":160,"sourceCodeEnd":195,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py#L160-L195","documentation":"The auxiliary scales '<prefix>.input_scale' and '<prefix>.output_scale' must each be a float-dtype scalar (shape ()). If either has a non-float dtype or any non-scalar shape, this error names the offending tensor.","triggerScenarios":"A prefix in the quantization_map whose input_scale/output_scale tensor is stored with shape (1,) or (n,) instead of (), or in an integer dtype.","commonSituations":"Export pipelines that unsqueeze scalars to (1,) (common when converting through numpy or other frameworks), dtype conversion to float64-int hybrids, or hand-authored scale tensors.","solutions":["Rewrite input_scale/output_scale as 0-d float tensors in the safetensors file (e.g. torch.tensor(v, dtype=torch.float32) with no dims)","If an export tool added a leading dim, squeeze it before saving","Re-export the checkpoint from the quantized model to regenerate correct scalar scales"],"exampleFix":"# before\ninput_scale = torch.tensor([0.0123])   # shape (1,)\n# after\ninput_scale = torch.tensor(0.0123, dtype=torch.float32)  # shape ()","handlingStrategy":"validation","validationCode":"for p in quantization_map:\n    for n in (f'{p}.input_scale', f'{p}.output_scale'):\n        sl = ckpt.get_slice(n)\n        if sl.get_dtype() not in ('F32','BF16','F16') or tuple(sl.get_shape()) != ():\n            raise SystemExit(f'{n}: expected float scalar, got {sl.get_dtype()} {sl.get_shape()}')","typeGuard":"def is_float_scalar(sl) -> bool:\n    return sl.get_dtype() in ('F32','F16','BF16') and tuple(sl.get_shape()) == ()","tryCatchPattern":"try:\n    cfg = inspect_quanto_int8_checkpoint(ckpt, mapper)\nexcept ValueError as e:\n    if 'must be a float scalar' in str(e):\n        raise SystemExit('Squeeze input/output scales to 0-d float tensors')\n    raise","preventionTips":["Save scales as torch scalar tensors, never length-1 arrays","Squeeze (1,) dims after numpy conversions before writing safetensors"],"tags":["quantization","quanto","scale","scalar-validation"],"backgroundTag":"invalid-tensor-dtype-shape","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}