{"record":{"id":"375d3149e5aad364","repo":"sgl-project/sglang","slug":"quanto-activation-quantization-is-not-supported-fo","errorCode":null,"errorMessage":"Quanto activation quantization is not supported for {prefix!r}","messagePattern":"Quanto activation quantization is not supported for (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py","lineNumber":135,"sourceCode":"        map_prefixes = set(quantization_map)\n        if data_prefixes != map_prefixes:\n            missing_map = data_prefixes - map_prefixes\n            missing_data = map_prefixes - data_prefixes\n            raise ValueError(\n                \"Quanto tensor/map prefixes do not match: \"\n                f\"missing metadata={sorted(missing_map)[:5]}, \"\n                f\"missing tensors={sorted(missing_data)[:5]}\"\n            )\n\n        mapped_prefixes: set[str] = set()\n        for prefix, quantization in quantization_map.items():\n            if quantization.get(\"weights\") != \"qint8\":\n                raise ValueError(\n                    f\"Unsupported Quanto weight type for {prefix!r}: \"\n                    f\"{quantization.get('weights')!r}\"\n                )\n            if quantization.get(\"activations\") != \"none\":\n                raise ValueError(\n                    f\"Quanto activation quantization is not supported for {prefix!r}\"\n                )\n\n            names = {\n                \"data\": f\"{prefix}.weight._data\",\n                \"scale\": f\"{prefix}.weight._scale\",\n                \"input\": f\"{prefix}.input_scale\",\n                \"output\": f\"{prefix}.output_scale\",\n            }\n            missing = set(names.values()) - checkpoint_keys\n            if missing:\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} is missing tensors: {sorted(missing)}\"\n                )\n            if f\"{prefix}.weight\" in checkpoint_keys:\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} contains both packed and dense weights\"\n                )","sourceCodeStart":117,"sourceCodeEnd":153,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py#L117-L153","documentation":"This loader supports only weight-only Quanto int8; activations must be \"none\" in the quantization_map. If any prefix declares activation quantization (e.g. qint8 activations), the checkpoint is rejected for that prefix.","triggerScenarios":"Loading a checkpoint quantized with dynamic/activation quantization (quantize(model, weights=qint8, activations=qint8)) and passing it through inspect_quanto_int8_checkpoint, e.g. via _get_encoder_quant_config.","commonSituations":"User quantizes with quanto's default example that includes activations, or a published checkpoint was made with weight+activation quantization while the runtime only implements weight-only int8 kernels.","solutions":["Re-quantize with weight-only mode: quantize weights to qint8 and leave activations unquantized (activations='none')","Use a backend that supports the checkpoint's activation quantization, or dequantize and re-quantize weight-only"],"exampleFix":"# before\nquantize(model, weights=qint8, activations=qint8)\n# after\nquantize(model, weights=qint8)  # activations stay none","handlingStrategy":"validation","validationCode":"act = {p: q for p, q in quantization_map.items() if q.get('activations') != 'none'}\nif act: raise SystemExit(f'activation-quantized layers present: {list(act)[:5]}')","typeGuard":"def is_weight_only(qmap: dict) -> bool:\n    return all(q.get('activations') in (None, 'none') for q in qmap.values())","tryCatchPattern":"try:\n    cfg = inspect_quanto_int8_checkpoint(ckpt, mapper)\nexcept ValueError as e:\n    if 'activation quantization is not supported' in str(e):\n        raise SystemExit('Re-export with weight-only quantization')\n    raise","preventionTips":["Quantize with quantize(model, weights=qint8) — never pass activations","Check the map's activations field before integrating third-party checkpoints"],"tags":["quantization","quanto","activations","weight-only"],"backgroundTag":"unsupported-quantization-scheme","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}