sgl-project/sglang · error · ValueError

Quanto quantization map must be a non-empty object

Error message

Quanto quantization map must be a non-empty object

What it means

After decoding, the quantization map must be a non-empty JSON object mapping layer prefixes to their quantization spec. An empty object, list, string, or null raises this error.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py:103

) -> QuantoInt8Config | None:
    """Validate a self-describing Quanto qint8 safetensors checkpoint."""

    with safe_open(file_path, framework="pt", device="cpu") as checkpoint:
        metadata = checkpoint.metadata() or {}
        if metadata.get("quantization_format") != "quanto":
            return None

        encoded_map = metadata.get("quantization_map_base64")
        if encoded_map is None:
            raise ValueError("Quanto checkpoint is missing quantization_map_base64")
        try:
            quantization_map = json.loads(
                base64.b64decode(encoded_map, validate=True).decode("utf-8")
            )
        except (ValueError, UnicodeDecodeError, json.JSONDecodeError) as error:
            raise ValueError("Invalid Quanto quantization_map_base64") from error
        if not isinstance(quantization_map, dict) or not quantization_map:
            raise ValueError("Quanto quantization map must be a non-empty object")
        if not all(
            isinstance(prefix, str) and isinstance(spec, dict)
            for prefix, spec in quantization_map.items()
        ):
            raise ValueError("Quanto quantization map entries must be named objects")

        checkpoint_keys = set(checkpoint.keys())
        data_suffix = ".weight._data"
        data_prefixes = {
            name.removesuffix(data_suffix)
            for name in checkpoint_keys
            if name.endswith(data_suffix)
        }
        map_prefixes = set(quantization_map)
        if data_prefixes != map_prefixes:
            missing_map = data_prefixes - map_prefixes
            missing_data = map_prefixes - data_prefixes
            raise ValueError(

View on GitHub (pinned to 0132848349)

Solutions

1. If nothing is quantized, remove the 'quantization_format': 'quanto' metadata instead of shipping an empty map
2. Re-export ensuring the map contains at least one entry: {prefix: {"weights": "int8", ...}}

Example fix

# before (empty map)
map_b64 = base64.b64encode(json.dumps({}).encode()).decode()

# after
map_b64 = base64.b64encode(json.dumps({"encoder.fc1": {"weights": "int8"}}).encode()).decode()

qmap = json.loads(base64.b64decode(meta["quantization_map_base64"]).decode())
if not isinstance(qmap, dict) or not qmap:
    meta.pop("quantization_format", None)  # treat as unquantized
    meta.pop("quantization_map_base64", None)

def is_non_empty_quant_map(m: object) -> bool:
    return isinstance(m, dict) and len(m) > 0

When it happens

Trigger: quantization_map_base64 decodes to '[]', '""', '{}', or 'null' — e.g. an empty dict was serialized when no layers were quantized, yet quantization_format was still set to 'quanto'.

Common situations: Exporting a model where quantization was skipped/no layers selected; a bug in the exporter serializing an uninitialized map.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Quanto quantization map entries must be named objects
• Quanto checkpoint is missing quantization_map_base64
• Invalid Quanto quantization_map_base64
• QuantoInt8Config must be constructed from safetensors metada
• Quanto tensor/map prefixes do not match: missing metadata={s

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3472ba4bee7ddab0. Report an issue: GitHub.