{"record":{"id":"d6e1dba00c75a37d","repo":"sgl-project/sglang","slug":"quanto-layer-prefix-r-needs-a-2d-i8-weight-got","errorCode":null,"errorMessage":"Quanto layer {prefix!r} needs a 2D I8 weight, got {data_slice.get_dtype()} {data_shape}","messagePattern":"Quanto layer (.+?) needs a 2D I8 weight, got (.+?) (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py","lineNumber":160,"sourceCode":"                \"input\": f\"{prefix}.input_scale\",\n                \"output\": f\"{prefix}.output_scale\",\n            }\n            missing = set(names.values()) - checkpoint_keys\n            if missing:\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} is missing tensors: {sorted(missing)}\"\n                )\n            if f\"{prefix}.weight\" in checkpoint_keys:\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} contains both packed and dense weights\"\n                )\n\n            data_slice = checkpoint.get_slice(names[\"data\"])\n            scale_slice = checkpoint.get_slice(names[\"scale\"])\n            data_shape = tuple(data_slice.get_shape())\n            scale_shape = tuple(scale_slice.get_shape())\n            if data_slice.get_dtype() != \"I8\" or len(data_shape) != 2:\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} needs a 2D I8 weight, got \"\n                    f\"{data_slice.get_dtype()} {data_shape}\"\n                )\n            if scale_slice.get_dtype() not in _FLOAT_DTYPES or scale_shape != (\n                data_shape[0],\n                1,\n            ):\n                raise ValueError(\n                    f\"Quanto layer {prefix!r} has incompatible scale \"\n                    f\"{scale_slice.get_dtype()} {scale_shape}\"\n                )\n            for scale_name in (names[\"input\"], names[\"output\"]):\n                scale = checkpoint.get_slice(scale_name)\n                if (\n                    scale.get_dtype() not in _FLOAT_DTYPES\n                    or tuple(scale.get_shape()) != ()\n                ):\n                    raise ValueError(","sourceCodeStart":142,"sourceCodeEnd":178,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py#L142-L178","documentation":"The packed weight tensor '<prefix>.weight._data' must be a 2D tensor of raw int8 dtype ('I8' in the safetensors file). This error reports the actual dtype and shape when either the dtype is not I8 or the tensor is not 2D.","triggerScenarios":"Loading a checkpoint where _data was saved as int32/int16/uint8, or as a higher-rank tensor (e.g. 3D conv weights or a reshaped/export artifact), via inspect_quanto_int8_checkpoint.","commonSituations":"Quantized 1D-conv/conv3d layers whose packed data isn't 2D, exporting through a format that widened the dtype (e.g. torch.save round-trip or safetensors convert), or hand-crafting _data tensors.","solutions":["Confirm data_slice.get_dtype()=='I8' and len(shape)==2 with safetensors before loading","Re-quantize so that the target layer is a Linear (2D) layer; exclude conv/other-rank layers from the quantization map","If dtype was widened during export, rewrite the tensor as int8 in the safetensors file"],"exampleFix":"# validation\nsl = ckpt.get_slice(f\"{p}.weight._data\")\nassert sl.get_dtype() == \"I8\" and len(sl.get_shape()) == 2, (sl.get_dtype(), sl.get_shape())","handlingStrategy":"validation","validationCode":"for p in quantization_map:\n    sl = ckpt.get_slice(f'{p}.weight._data')\n    if sl.get_dtype() != 'I8' or len(sl.get_shape()) != 2:\n        raise SystemExit(f'{p}: bad _data {sl.get_dtype()} {sl.get_shape()}')","typeGuard":"def is_valid_i8_2d(sl) -> bool:\n    return sl.get_dtype() == 'I8' and len(sl.get_shape()) == 2","tryCatchPattern":"try:\n    cfg = inspect_quanto_int8_checkpoint(ckpt, mapper)\nexcept ValueError as e:\n    if 'needs a 2D I8 weight' in str(e):\n        raise SystemExit('Re-export _data as int8 2D; exclude non-linear layers from map')\n    raise","preventionTips":["Only quantize Linear layers for this path","Avoid round-tripping safetensors through formats that widen dtypes"],"tags":["quantization","quanto","dtype","shape-validation"],"backgroundTag":"invalid-tensor-dtype-shape","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}