{"record":{"id":"d7c3b2ec2613ac5c","repo":"sgl-project/sglang","slug":"quanto-tensor-map-prefixes-do-not-match-missing-m","errorCode":null,"errorMessage":"Quanto tensor/map prefixes do not match: missing metadata={sorted(missing_map)[:5]}, missing tensors={sorted(missing_data)[:5]}","messagePattern":"Quanto tensor/map prefixes do not match: missing metadata=(.+?), missing tensors=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py","lineNumber":121,"sourceCode":"            raise ValueError(\"Quanto quantization map must be a non-empty object\")\n        if not all(\n            isinstance(prefix, str) and isinstance(spec, dict)\n            for prefix, spec in quantization_map.items()\n        ):\n            raise ValueError(\"Quanto quantization map entries must be named objects\")\n\n        checkpoint_keys = set(checkpoint.keys())\n        data_suffix = \".weight._data\"\n        data_prefixes = {\n            name.removesuffix(data_suffix)\n            for name in checkpoint_keys\n            if name.endswith(data_suffix)\n        }\n        map_prefixes = set(quantization_map)\n        if data_prefixes != map_prefixes:\n            missing_map = data_prefixes - map_prefixes\n            missing_data = map_prefixes - data_prefixes\n            raise ValueError(\n                \"Quanto tensor/map prefixes do not match: \"\n                f\"missing metadata={sorted(missing_map)[:5]}, \"\n                f\"missing tensors={sorted(missing_data)[:5]}\"\n            )\n\n        mapped_prefixes: set[str] = set()\n        for prefix, quantization in quantization_map.items():\n            if quantization.get(\"weights\") != \"qint8\":\n                raise ValueError(\n                    f\"Unsupported Quanto weight type for {prefix!r}: \"\n                    f\"{quantization.get('weights')!r}\"\n                )\n            if quantization.get(\"activations\") != \"none\":\n                raise ValueError(\n                    f\"Quanto activation quantization is not supported for {prefix!r}\"\n                )\n\n            names = {","sourceCodeStart":103,"sourceCodeEnd":139,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py#L103-L139","documentation":"Raised by inspect_quanto_int8_checkpoint when the set of checkpoint tensor prefixes (keys ending in the Quanto _data suffix) does not match the set of prefixes listed in the checkpoint's quantization_map metadata. The library enforces that every layer declared as quantized actually has its packed tensors present, and vice versa, so a partially quantized or corrupted Quanto checkpoint is rejected before loading.","triggerScenarios":"Loading a checkpoint whose quantization_map (e.g. quanto_metadata.json / map entry) lists prefixes with no corresponding '<prefix>.weight._data' tensors, or tensors exist for prefixes absent from the map. Triggered via inspect_quanto_int8_checkpoint(), directly or through _get_encoder_quant_config.","commonSituations":"Manually edited or pruned checkpoints, saving with an older/newer quanto version whose map format differs, removing layers from the safetensors but not the metadata, or copying only some tensors when sharding/merging.","solutions":["Inspect the reported missing prefixes (first 5 are in the message) and fix the quantization map or re-export the checkpoint with the same quanto version","Verify every prefix in quantization_map has keys '<p>.weight._data', '<p>.weight._scale', '<p>.input_scale', '<p>.output_scale' in the checkpoint","Re-quantize the model from the original FP16/BF16 weights so map and tensors are generated together","If intentional mixed precision, exclude non-quantized layers from the map instead of deleting tensors"],"exampleFix":"// before: map lists 'encoder.layers.0.self_attn' but tensors were deleted\n// after: re-export with quanto so map and tensors stay consistent\nmodel.save_pretrained(out_dir)  # tensors + quanto metadata written atomically","handlingStrategy":"validation","validationCode":"data_prefixes = {k.rsplit('.weight._data',1)[0] for k in ckpt_keys if k.endswith('.weight._data')}\nmap_prefixes = set(quantization_map)\nif data_prefixes != map_prefixes:\n    raise SystemExit(f'map/tensor mismatch: {data_prefixes ^ map_prefixes}')","typeGuard":null,"tryCatchPattern":"try:\n    cfg = inspect_quanto_int8_checkpoint(ckpt, mapper)\nexcept ValueError as e:\n    if 'prefixes do not match' in str(e):\n        raise SystemExit(f'Bad Quanto checkpoint: {e}; re-export with matching quanto version')\n    raise","preventionTips":["Always generate the quantization map and tensors in the same quanto export run","Validate map-vs-keys with safetensors before starting a long server launch"],"tags":["quantization","quanto","checkpoint","int8","multimodal"],"backgroundTag":"checkpoint-metadata-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}