sgl-project/sglang · error · ValueError

Quanto layer {prefix!r} is missing tensors: {sorted(missing)

Error message

Quanto layer {prefix!r} is missing tensors: {sorted(missing)}

What it means

For every prefix in the quantization_map the checkpoint must contain the four Quanto tensors: '<prefix>.weight._data', '<prefix>.weight._scale', '<prefix>.input_scale', '<prefix>.output_scale'. If any are absent from the checkpoint keys, this error lists the missing tensor names.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py:147

            if quantization.get("weights") != "qint8":
                raise ValueError(
                    f"Unsupported Quanto weight type for {prefix!r}: "
                    f"{quantization.get('weights')!r}"
                )
            if quantization.get("activations") != "none":
                raise ValueError(
                    f"Quanto activation quantization is not supported for {prefix!r}"
                )

            names = {
                "data": f"{prefix}.weight._data",
                "scale": f"{prefix}.weight._scale",
                "input": f"{prefix}.input_scale",
                "output": f"{prefix}.output_scale",
            }
            missing = set(names.values()) - checkpoint_keys
            if missing:
                raise ValueError(
                    f"Quanto layer {prefix!r} is missing tensors: {sorted(missing)}"
                )
            if f"{prefix}.weight" in checkpoint_keys:
                raise ValueError(
                    f"Quanto layer {prefix!r} contains both packed and dense weights"
                )

            data_slice = checkpoint.get_slice(names["data"])
            scale_slice = checkpoint.get_slice(names["scale"])
            data_shape = tuple(data_slice.get_shape())
            scale_shape = tuple(scale_slice.get_shape())
            if data_slice.get_dtype() != "I8" or len(data_shape) != 2:
                raise ValueError(
                    f"Quanto layer {prefix!r} needs a 2D I8 weight, got "
                    f"{data_slice.get_dtype()} {data_shape}"
                )
            if scale_slice.get_dtype() not in _FLOAT_DTYPES or scale_shape != (
                data_shape[0],

View on GitHub (pinned to 0132848349)

Solutions

1. Check the listed missing tensor names and re-export the checkpoint with a quanto version that writes all four tensors per layer
2. If the source model truly lacks per-layer input/output scales, regenerate them during quantization rather than hand-editing the map
3. Verify checkpoint_keys with safetensors.safe_open before calling inspect_quanto_int8_checkpoint

Example fix

# validation
from safetensors import safe_open
with safe_open(file, framework="pt") as f:
    keys = set(f.keys())
need = {f"{p}.weight._data", f"{p}.weight._scale", f"{p}.input_scale", f"{p}.output_scale"}
assert need <= keys

need = lambda p: {f'{p}.weight._data', f'{p}.weight._scale', f'{p}.input_scale', f'{p}.output_scale'}
missing = {n for p in quantization_map for n in need(p) if n not in ckpt_keys}
if missing: raise SystemExit(f'missing tensors: {sorted(missing)[:5]}')

try:
    cfg = inspect_quanto_int8_checkpoint(ckpt, mapper)
except ValueError as e:
    if 'missing tensors' in str(e):
        raise SystemExit(f'incomplete Quanto checkpoint: {e}')
    raise

When it happens

Trigger: A quantization_map prefix whose safetensors file lacks one or more of the four required keys — e.g. only _data/_scale saved but not the input/output scales, or a key renamed during export.

Common situations: Checkpoints saved with an older quanto layout that omitted input_scale/output_scale, manual key renaming, tensor pruning, or partial file merge where some safetensors shards were dropped.

Related errors

• Quanto tensor/map prefixes do not match: missing metadata={s
• Quanto layer {prefix!r} contains both packed and dense weigh
• SGLang diffusion currently supports AutoRound auto_gptq chec
• AutoRound fused module {target!r} has inconsistent shard con
• Parameter {param_name} not found in the model.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/93ac54b3dde0bbf7. Report an issue: GitHub.