{"record":{"id":"93ac54b3dde0bbf7","repo":"sgl-project/sglang","slug":"quanto-layer-prefix-r-is-missing-tensors-sorte","errorCode":null,"errorMessage":"Quanto layer {prefix!r} is missing tensors: {sorted(missing)}","messagePattern":"Quanto layer (.+?) is missing tensors: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py","lineNumber":147,"sourceCode":"            if quantization.get(\"weights\") != \"qint8\":\n                raise ValueError(\n                    f\"Unsupported Quanto weight type for {prefix!r}: \"\n                    f\"{quantization.get('weights')!r}\"\n                )\n            if quantization.get(\"activations\") != \"none\":\n                raise ValueError(\n                    f\"Quanto activation quantization is not supported for {prefix!r}\"\n                )\n\n            names = {\n                \"data\": f\"{prefix}.weight._data\",\n                \"scale\": f\"{prefix}.weight._scale\",\n                \"input\": f\"{prefix}.input_scale\",\n                \"output\": f\"{prefix}.output_scale\",\n            }\n            missing = set(names.values()) - checkpoint_keys\n            if missing:\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} is missing tensors: {sorted(missing)}\"\n                )\n            if f\"{prefix}.weight\" in checkpoint_keys:\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} contains both packed and dense weights\"\n                )\n\n            data_slice = checkpoint.get_slice(names[\"data\"])\n            scale_slice = checkpoint.get_slice(names[\"scale\"])\n            data_shape = tuple(data_slice.get_shape())\n            scale_shape = tuple(scale_slice.get_shape())\n            if data_slice.get_dtype() != \"I8\" or len(data_shape) != 2:\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} needs a 2D I8 weight, got \"\n                    f\"{data_slice.get_dtype()} {data_shape}\"\n                )\n            if scale_slice.get_dtype() not in _FLOAT_DTYPES or scale_shape != (\n                data_shape[0],","sourceCodeStart":129,"sourceCodeEnd":165,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py#L129-L165","documentation":"For every prefix in the quantization_map the checkpoint must contain the four Quanto tensors: '<prefix>.weight._data', '<prefix>.weight._scale', '<prefix>.input_scale', '<prefix>.output_scale'. If any are absent from the checkpoint keys, this error lists the missing tensor names.","triggerScenarios":"A quantization_map prefix whose safetensors file lacks one or more of the four required keys — e.g. only _data/_scale saved but not the input/output scales, or a key renamed during export.","commonSituations":"Checkpoints saved with an older quanto layout that omitted input_scale/output_scale, manual key renaming, tensor pruning, or partial file merge where some safetensors shards were dropped.","solutions":["Check the listed missing tensor names and re-export the checkpoint with a quanto version that writes all four tensors per layer","If the source model truly lacks per-layer input/output scales, regenerate them during quantization rather than hand-editing the map","Verify checkpoint_keys with safetensors.safe_open before calling inspect_quanto_int8_checkpoint"],"exampleFix":"# validation\nfrom safetensors import safe_open\nwith safe_open(file, framework=\"pt\") as f:\n    keys = set(f.keys())\nneed = {f\"{p}.weight._data\", f\"{p}.weight._scale\", f\"{p}.input_scale\", f\"{p}.output_scale\"}\nassert need <= keys","handlingStrategy":"validation","validationCode":"need = lambda p: {f'{p}.weight._data', f'{p}.weight._scale', f'{p}.input_scale', f'{p}.output_scale'}\nmissing = {n for p in quantization_map for n in need(p) if n not in ckpt_keys}\nif missing: raise SystemExit(f'missing tensors: {sorted(missing)[:5]}')","typeGuard":null,"tryCatchPattern":"try:\n    cfg = inspect_quanto_int8_checkpoint(ckpt, mapper)\nexcept ValueError as e:\n    if 'missing tensors' in str(e):\n        raise SystemExit(f'incomplete Quanto checkpoint: {e}')\n    raise","preventionTips":["Never delete individual tensors from a quantized checkpoint","Verify all four per-layer keys exist after merging or resharding safetensors"],"tags":["quantization","quanto","missing-tensor","checkpoint"],"backgroundTag":"checkpoint-metadata-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}