sgl-project/sglang · error · ValueError

Invalid Quanto quantization_map_base64

Error message

Invalid Quanto quantization_map_base64

What it means

The quantization_map_base64 metadata field failed to decode: it is not valid base64, not valid UTF-8 after decoding, or the decoded text is not JSON. The decode+parse is wrapped so any of these corruption modes surfaces as this single error.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py:101

    file_path: str,
    param_name_mapper: Callable[[str], str] | None = None,
) -> QuantoInt8Config | None:
    """Validate a self-describing Quanto qint8 safetensors checkpoint."""

    with safe_open(file_path, framework="pt", device="cpu") as checkpoint:
        metadata = checkpoint.metadata() or {}
        if metadata.get("quantization_format") != "quanto":
            return None

        encoded_map = metadata.get("quantization_map_base64")
        if encoded_map is None:
            raise ValueError("Quanto checkpoint is missing quantization_map_base64")
        try:
            quantization_map = json.loads(
                base64.b64decode(encoded_map, validate=True).decode("utf-8")
            )
        except (ValueError, UnicodeDecodeError, json.JSONDecodeError) as error:
            raise ValueError("Invalid Quanto quantization_map_base64") from error
        if not isinstance(quantization_map, dict) or not quantization_map:
            raise ValueError("Quanto quantization map must be a non-empty object")
        if not all(
            isinstance(prefix, str) and isinstance(spec, dict)
            for prefix, spec in quantization_map.items()
        ):
            raise ValueError("Quanto quantization map entries must be named objects")

        checkpoint_keys = set(checkpoint.keys())
        data_suffix = ".weight._data"
        data_prefixes = {
            name.removesuffix(data_suffix)
            for name in checkpoint_keys
            if name.endswith(data_suffix)
        }
        map_prefixes = set(quantization_map)
        if data_prefixes != map_prefixes:
            missing_map = data_prefixes - map_prefixes

View on GitHub (pinned to 0132848349)

Solutions

  1. Regenerate the map with base64.b64encode(json.dumps(map).encode()).decode()
  2. Verify round-trip: json.loads(base64.b64decode(value, validate=True)) succeeds before writing
  3. Check for urlsafe vs standard base64 and transcribe '-'/'_' to '+'/'/' if needed

Example fix

# before (corrupted / non-base64)
metadata["quantization_map_base64"] = "{not base64!!}"

# after
import base64, json
metadata["quantization_map_base64"] = base64.b64encode(
    json.dumps({"encoder.layers.0": {"weights": "int8"}}).encode("utf-8")
).decode("ascii")
Defensive patterns

Strategy: validation

Validate before calling

import base64, json
def map_decodes(b64: str) -> bool:
    try:
        json.loads(base64.b64decode(b64, validate=True).decode("utf-8"))
        return True
    except Exception:
        return False

Type guard

def is_valid_quanto_map_b64(value: object) -> bool:
    return (
        isinstance(value, str)
        and json.loads(base64.b64decode(value, validate=True).decode("utf-8")) is not None
    ) if isinstance(value, str) and _decodes(value) else False

Try / catch

try:
    cfg = inspect_quanto_int8_checkpoint(ckpt)
except ValueError as e:
    if "quantization_map_base64" in str(e):
        raise RuntimeError("corrupt quanto metadata; re-export checkpoint") from e
    raise

Prevention

When it happens

Trigger: A quantization_map_base64 header containing characters outside the base64 alphabet (validate=True rejects them), stray whitespace/newlines, or double-encoded JSON.

Common situations: Metadata edited or re-serialized by a converter tool; base64 with padding stripped; JSON that was urlsafe-base64 encoded instead of standard base64.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b2b8f1712573157c. Report an issue: GitHub.