sgl-project/sglang · error · ValueError

Quanto quantization map entries must be named objects

Error message

Quanto quantization map entries must be named objects

What it means

Every entry in the decoded quantization map must be a JSON object keyed by a string layer prefix with a dict value (the layer's quantization spec). Any non-string key or non-dict value (e.g. a string, list, or number) fails this check.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py:108

        if metadata.get("quantization_format") != "quanto":
            return None

        encoded_map = metadata.get("quantization_map_base64")
        if encoded_map is None:
            raise ValueError("Quanto checkpoint is missing quantization_map_base64")
        try:
            quantization_map = json.loads(
                base64.b64decode(encoded_map, validate=True).decode("utf-8")
            )
        except (ValueError, UnicodeDecodeError, json.JSONDecodeError) as error:
            raise ValueError("Invalid Quanto quantization_map_base64") from error
        if not isinstance(quantization_map, dict) or not quantization_map:
            raise ValueError("Quanto quantization map must be a non-empty object")
        if not all(
            isinstance(prefix, str) and isinstance(spec, dict)
            for prefix, spec in quantization_map.items()
        ):
            raise ValueError("Quanto quantization map entries must be named objects")

        checkpoint_keys = set(checkpoint.keys())
        data_suffix = ".weight._data"
        data_prefixes = {
            name.removesuffix(data_suffix)
            for name in checkpoint_keys
            if name.endswith(data_suffix)
        }
        map_prefixes = set(quantization_map)
        if data_prefixes != map_prefixes:
            missing_map = data_prefixes - map_prefixes
            missing_data = map_prefixes - data_prefixes
            raise ValueError(
                "Quanto tensor/map prefixes do not match: "
                f"missing metadata={sorted(missing_map)[:5]}, "
                f"missing tensors={sorted(missing_data)[:5]}"
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Normalize each entry to {prefix: {"weights": "int8"}} style spec dicts
2. Validate your map with the same check (str key + dict value) before serializing

Example fix

# before
{"encoder.fc1": "int8"}

# after
{"encoder.fc1": {"weights": "int8", "activation": null}}

ok = isinstance(qmap, dict) and qmap and all(
    isinstance(k, str) and isinstance(v, dict) for k, v in qmap.items()
)
if not ok:
    raise RuntimeError("malformed quantization map; expected {prefix: spec_dict}")

def is_wellformed_quant_map(m: object) -> bool:
    return isinstance(m, dict) and bool(m) and all(
        isinstance(k, str) and isinstance(v, dict) for k, v in m.items()
    )

When it happens

Trigger: A map like {"encoder.fc1": "int8"} (value is a string instead of a spec dict) or {"0": ["int8"]}; malformed maps from custom exporters that use shorthand values.

Common situations: Hand-written or converted maps using abbreviated layer specs; exporters that store the dtype string directly instead of a spec object like {"weights": "int8", "activation": null}.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Quanto quantization map must be a non-empty object
• Quanto checkpoint is missing quantization_map_base64
• Invalid Quanto quantization_map_base64
• QuantoInt8Config must be constructed from safetensors metada
• Quanto tensor/map prefixes do not match: missing metadata={s

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/935482d4e5c75d13. Report an issue: GitHub.