{"record":{"id":"ebae301945ad84f0","repo":"sgl-project/sglang","slug":"quanto-layer-prefix-r-contains-both-packed-and-d","errorCode":null,"errorMessage":"Quanto layer {prefix!r} contains both packed and dense weights","messagePattern":"Quanto layer (.+?) contains both packed and dense weights","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py","lineNumber":151,"sourceCode":"                )\n            if quantization.get(\"activations\") != \"none\":\n                raise ValueError(\n                    f\"Quanto activation quantization is not supported for {prefix!r}\"\n                )\n\n            names = {\n                \"data\": f\"{prefix}.weight._data\",\n                \"scale\": f\"{prefix}.weight._scale\",\n                \"input\": f\"{prefix}.input_scale\",\n                \"output\": f\"{prefix}.output_scale\",\n            }\n            missing = set(names.values()) - checkpoint_keys\n            if missing:\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} is missing tensors: {sorted(missing)}\"\n                )\n            if f\"{prefix}.weight\" in checkpoint_keys:\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} contains both packed and dense weights\"\n                )\n\n            data_slice = checkpoint.get_slice(names[\"data\"])\n            scale_slice = checkpoint.get_slice(names[\"scale\"])\n            data_shape = tuple(data_slice.get_shape())\n            scale_shape = tuple(scale_slice.get_shape())\n            if data_slice.get_dtype() != \"I8\" or len(data_shape) != 2:\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} needs a 2D I8 weight, got \"\n                    f\"{data_slice.get_dtype()} {data_shape}\"\n                )\n            if scale_slice.get_dtype() not in _FLOAT_DTYPES or scale_shape != (\n                data_shape[0],\n                1,\n            ):\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} has incompatible scale \"","sourceCodeStart":133,"sourceCodeEnd":169,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py#L133-L169","documentation":"A layer declared quantized in the quantization_map contains both the packed Quanto tensors ('<prefix>.weight._data' etc.) and the original dense weight key '<prefix>.weight'. This ambiguity is rejected because it's unclear which representation should be loaded.","triggerScenarios":"Checkpoint that kept the original FP16/BF16 '<prefix>.weight' tensors alongside the quanto-packed '_data'/'_scale' tensors for the same prefix listed in quantization_map.","commonSituations":"Saving the quantized model without freezing/removing original weights, merging safetensors shards from the base model and the quantized export, or a export tool that writes both representations.","solutions":["Delete the dense '<prefix>.weight' entries from the checkpoint for every prefix in the map (keep only _data/_scale/scales)","Re-export with quanto after freeze()+freeze_qparams so only packed tensors are serialized"],"exampleFix":"# before: shard contains both\n# 'enc.0.weight' AND 'enc.0.weight._data' / 'enc.0.weight._scale'\n# after: keep only\n# 'enc.0.weight._data', 'enc.0.weight._scale', 'enc.0.input_scale', 'enc.0.output_scale'","handlingStrategy":"validation","validationCode":"dense = [p for p in quantization_map if f'{p}.weight' in ckpt_keys]\nif dense: raise SystemExit(f'dense weights still present for: {dense[:5]}')","typeGuard":null,"tryCatchPattern":"try:\n    cfg = inspect_quanto_int8_checkpoint(ckpt, mapper)\nexcept ValueError as e:\n    if 'both packed and dense weights' in str(e):\n        raise SystemExit('Strip original dense weights from the checkpoint')\n    raise","preventionTips":["Call quanto freeze() before save_pretrained so dense weights are dropped","Don't merge base-model shards into a quantized export"],"tags":["quantization","quanto","duplicate-weights","checkpoint"],"backgroundTag":"checkpoint-metadata-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}