sgl-project/sglang · error · ValueError

Quanto activation quantization is not supported for {prefix!

Error message

Quanto activation quantization is not supported for {prefix!r}

What it means

This loader supports only weight-only Quanto int8; activations must be "none" in the quantization_map. If any prefix declares activation quantization (e.g. qint8 activations), the checkpoint is rejected for that prefix.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py:135

        map_prefixes = set(quantization_map)
        if data_prefixes != map_prefixes:
            missing_map = data_prefixes - map_prefixes
            missing_data = map_prefixes - data_prefixes
            raise ValueError(
                "Quanto tensor/map prefixes do not match: "
                f"missing metadata={sorted(missing_map)[:5]}, "
                f"missing tensors={sorted(missing_data)[:5]}"
            )

        mapped_prefixes: set[str] = set()
        for prefix, quantization in quantization_map.items():
            if quantization.get("weights") != "qint8":
                raise ValueError(
                    f"Unsupported Quanto weight type for {prefix!r}: "
                    f"{quantization.get('weights')!r}"
                )
            if quantization.get("activations") != "none":
                raise ValueError(
                    f"Quanto activation quantization is not supported for {prefix!r}"
                )

            names = {
                "data": f"{prefix}.weight._data",
                "scale": f"{prefix}.weight._scale",
                "input": f"{prefix}.input_scale",
                "output": f"{prefix}.output_scale",
            }
            missing = set(names.values()) - checkpoint_keys
            if missing:
                raise ValueError(
                    f"Quanto layer {prefix!r} is missing tensors: {sorted(missing)}"
                )
            if f"{prefix}.weight" in checkpoint_keys:
                raise ValueError(
                    f"Quanto layer {prefix!r} contains both packed and dense weights"
                )

View on GitHub (pinned to 0132848349)

Solutions

Re-quantize with weight-only mode: quantize weights to qint8 and leave activations unquantized (activations='none')
Use a backend that supports the checkpoint's activation quantization, or dequantize and re-quantize weight-only

Example fix

# before
quantize(model, weights=qint8, activations=qint8)
# after
quantize(model, weights=qint8)  # activations stay none

Defensive patterns

Strategy: validation

Validate before calling

act = {p: q for p, q in quantization_map.items() if q.get('activations') != 'none'}
if act: raise SystemExit(f'activation-quantized layers present: {list(act)[:5]}')

Type guard

def is_weight_only(qmap: dict) -> bool:
    return all(q.get('activations') in (None, 'none') for q in qmap.values())

Try / catch

try:
    cfg = inspect_quanto_int8_checkpoint(ckpt, mapper)
except ValueError as e:
    if 'activation quantization is not supported' in str(e):
        raise SystemExit('Re-export with weight-only quantization')
    raise

Prevention

Quantize with quantize(model, weights=qint8) — never pass activations
Check the map's activations field before integrating third-party checkpoints

When it happens

Trigger: Loading a checkpoint quantized with dynamic/activation quantization (quantize(model, weights=qint8, activations=qint8)) and passing it through inspect_quanto_int8_checkpoint, e.g. via _get_encoder_quant_config.

Common situations: User quantizes with quanto's default example that includes activations, or a published checkpoint was made with weight+activation quantization while the runtime only implements weight-only int8 kernels.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/375d3149e5aad364. Report an issue: GitHub.