sgl-project/sglang · error · ValueError

Quanto checkpoint is missing quantization_map_base64

Error message

Quanto checkpoint is missing quantization_map_base64

What it means

inspect_quanto_int8_checkpoint found a safetensors file whose metadata has quantization_format == 'quanto' but no quantization_map_base64 key. The per-layer quantization map (which layers are int8-quantized) is mandatory for quanto checkpoints in this runtime.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py:95

            return unquantized_method()
        self.selected.add(prefix)
        return QuantoInt8LinearMethod()


def inspect_quanto_int8_checkpoint(
    file_path: str,
    param_name_mapper: Callable[[str], str] | None = None,
) -> QuantoInt8Config | None:
    """Validate a self-describing Quanto qint8 safetensors checkpoint."""

    with safe_open(file_path, framework="pt", device="cpu") as checkpoint:
        metadata = checkpoint.metadata() or {}
        if metadata.get("quantization_format") != "quanto":
            return None

        encoded_map = metadata.get("quantization_map_base64")
        if encoded_map is None:
            raise ValueError("Quanto checkpoint is missing quantization_map_base64")
        try:
            quantization_map = json.loads(
                base64.b64decode(encoded_map, validate=True).decode("utf-8")
            )
        except (ValueError, UnicodeDecodeError, json.JSONDecodeError) as error:
            raise ValueError("Invalid Quanto quantization_map_base64") from error
        if not isinstance(quantization_map, dict) or not quantization_map:
            raise ValueError("Quanto quantization map must be a non-empty object")
        if not all(
            isinstance(prefix, str) and isinstance(spec, dict)
            for prefix, spec in quantization_map.items()
        ):
            raise ValueError("Quanto quantization map entries must be named objects")

        checkpoint_keys = set(checkpoint.keys())
        data_suffix = ".weight._data"
        data_prefixes = {
            name.removesuffix(data_suffix)

View on GitHub (pinned to 0132848349)

Solutions

1. Re-export the checkpoint with a quanto version that writes quantization_map_base64 metadata
2. Re-attach the base64-encoded JSON map to the safetensors header
3. If the model is actually unquantized, remove the quantization_format='quanto' metadata key

Example fix

# before
metadata = {"quantization_format": "quanto"}  # missing map -> ValueError

# after
import base64, json
metadata = {
  "quantization_format": "quanto",
  "quantization_map_base64": base64.b64encode(json.dumps({"encoder.layers.0": {"weights": "int8"}}).encode()).decode(),
}

meta = checkpoint.metadata() or {}
if meta.get("quantization_format") == "quanto" and "quantization_map_base64" not in meta:
    raise RuntimeError("quanto checkpoint lacks quantization map; re-export it")

def is_complete_quanto_metadata(meta: dict | None) -> bool:
    meta = meta or {}
    return meta.get("quantization_format") != "quanto" or "quantization_map_base64" in meta

When it happens

Trigger: Opening a quanto-flavored safetensors file whose header lacks 'quantization_map_base64' — e.g. exported by an older quanto version or hand-stripped metadata.

Common situations: Version mismatch between the quanto exporter and this runtime; merging/converting checkpoints with tools that drop unknown metadata fields.

Related errors

• Invalid Quanto quantization_map_base64
• Quanto quantization map must be a non-empty object
• Quanto quantization map entries must be named objects
• QuantoInt8Config must be constructed from safetensors metada
• Quanto tensor/map prefixes do not match: missing metadata={s

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1da24917cc84dd93. Report an issue: GitHub.