sgl-project/sglang · error · ValueError

Quanto tensor/map prefixes do not match: missing metadata={s

Error message

Quanto tensor/map prefixes do not match: missing metadata={sorted(missing_map)[:5]}, missing tensors={sorted(missing_data)[:5]}

What it means

Raised by inspect_quanto_int8_checkpoint when the set of checkpoint tensor prefixes (keys ending in the Quanto _data suffix) does not match the set of prefixes listed in the checkpoint's quantization_map metadata. The library enforces that every layer declared as quantized actually has its packed tensors present, and vice versa, so a partially quantized or corrupted Quanto checkpoint is rejected before loading.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py:121

            raise ValueError("Quanto quantization map must be a non-empty object")
        if not all(
            isinstance(prefix, str) and isinstance(spec, dict)
            for prefix, spec in quantization_map.items()
        ):
            raise ValueError("Quanto quantization map entries must be named objects")

        checkpoint_keys = set(checkpoint.keys())
        data_suffix = ".weight._data"
        data_prefixes = {
            name.removesuffix(data_suffix)
            for name in checkpoint_keys
            if name.endswith(data_suffix)
        }
        map_prefixes = set(quantization_map)
        if data_prefixes != map_prefixes:
            missing_map = data_prefixes - map_prefixes
            missing_data = map_prefixes - data_prefixes
            raise ValueError(
                "Quanto tensor/map prefixes do not match: "
                f"missing metadata={sorted(missing_map)[:5]}, "
                f"missing tensors={sorted(missing_data)[:5]}"
            )

        mapped_prefixes: set[str] = set()
        for prefix, quantization in quantization_map.items():
            if quantization.get("weights") != "qint8":
                raise ValueError(
                    f"Unsupported Quanto weight type for {prefix!r}: "
                    f"{quantization.get('weights')!r}"
                )
            if quantization.get("activations") != "none":
                raise ValueError(
                    f"Quanto activation quantization is not supported for {prefix!r}"
                )

            names = {

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect the reported missing prefixes (first 5 are in the message) and fix the quantization map or re-export the checkpoint with the same quanto version
2. Verify every prefix in quantization_map has keys '<p>.weight._data', '<p>.weight._scale', '<p>.input_scale', '<p>.output_scale' in the checkpoint
3. Re-quantize the model from the original FP16/BF16 weights so map and tensors are generated together
4. If intentional mixed precision, exclude non-quantized layers from the map instead of deleting tensors

Example fix

// before: map lists 'encoder.layers.0.self_attn' but tensors were deleted
// after: re-export with quanto so map and tensors stay consistent
model.save_pretrained(out_dir)  # tensors + quanto metadata written atomically

data_prefixes = {k.rsplit('.weight._data',1)[0] for k in ckpt_keys if k.endswith('.weight._data')}
map_prefixes = set(quantization_map)
if data_prefixes != map_prefixes:
    raise SystemExit(f'map/tensor mismatch: {data_prefixes ^ map_prefixes}')

try:
    cfg = inspect_quanto_int8_checkpoint(ckpt, mapper)
except ValueError as e:
    if 'prefixes do not match' in str(e):
        raise SystemExit(f'Bad Quanto checkpoint: {e}; re-export with matching quanto version')
    raise

When it happens

Trigger: Loading a checkpoint whose quantization_map (e.g. quanto_metadata.json / map entry) lists prefixes with no corresponding '<prefix>.weight._data' tensors, or tensors exist for prefixes absent from the map. Triggered via inspect_quanto_int8_checkpoint(), directly or through _get_encoder_quant_config.

Common situations: Manually edited or pruned checkpoints, saving with an older/newer quanto version whose map format differs, removing layers from the safetensors but not the metadata, or copying only some tensors when sharding/merging.

Related errors

• Unsupported Quanto weight type for {prefix!r}: {quantization
• Quanto layer {prefix!r} is missing tensors: {sorted(missing)
• Quanto layer {prefix!r} contains both packed and dense weigh
• A GGUF encoder checkpoint cannot be combined with a second q
• SGLang diffusion currently supports AutoRound auto_gptq chec

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d7c3b2ec2613ac5c. Report an issue: GitHub.