sgl-project/sglang · error · ValueError

Quanto layer {prefix!r} contains both packed and dense weigh

Error message

Quanto layer {prefix!r} contains both packed and dense weights

What it means

A layer declared quantized in the quantization_map contains both the packed Quanto tensors ('<prefix>.weight._data' etc.) and the original dense weight key '<prefix>.weight'. This ambiguity is rejected because it's unclear which representation should be loaded.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py:151

                )
            if quantization.get("activations") != "none":
                raise ValueError(
                    f"Quanto activation quantization is not supported for {prefix!r}"
                )

            names = {
                "data": f"{prefix}.weight._data",
                "scale": f"{prefix}.weight._scale",
                "input": f"{prefix}.input_scale",
                "output": f"{prefix}.output_scale",
            }
            missing = set(names.values()) - checkpoint_keys
            if missing:
                raise ValueError(
                    f"Quanto layer {prefix!r} is missing tensors: {sorted(missing)}"
                )
            if f"{prefix}.weight" in checkpoint_keys:
                raise ValueError(
                    f"Quanto layer {prefix!r} contains both packed and dense weights"
                )

            data_slice = checkpoint.get_slice(names["data"])
            scale_slice = checkpoint.get_slice(names["scale"])
            data_shape = tuple(data_slice.get_shape())
            scale_shape = tuple(scale_slice.get_shape())
            if data_slice.get_dtype() != "I8" or len(data_shape) != 2:
                raise ValueError(
                    f"Quanto layer {prefix!r} needs a 2D I8 weight, got "
                    f"{data_slice.get_dtype()} {data_shape}"
                )
            if scale_slice.get_dtype() not in _FLOAT_DTYPES or scale_shape != (
                data_shape[0],
                1,
            ):
                raise ValueError(
                    f"Quanto layer {prefix!r} has incompatible scale "

View on GitHub (pinned to 0132848349)

Solutions

Delete the dense '<prefix>.weight' entries from the checkpoint for every prefix in the map (keep only _data/_scale/scales)
Re-export with quanto after freeze()+freeze_qparams so only packed tensors are serialized

Example fix

# before: shard contains both
# 'enc.0.weight' AND 'enc.0.weight._data' / 'enc.0.weight._scale'
# after: keep only
# 'enc.0.weight._data', 'enc.0.weight._scale', 'enc.0.input_scale', 'enc.0.output_scale'

Defensive patterns

Strategy: validation

Validate before calling

dense = [p for p in quantization_map if f'{p}.weight' in ckpt_keys]
if dense: raise SystemExit(f'dense weights still present for: {dense[:5]}')

Try / catch

try:
    cfg = inspect_quanto_int8_checkpoint(ckpt, mapper)
except ValueError as e:
    if 'both packed and dense weights' in str(e):
        raise SystemExit('Strip original dense weights from the checkpoint')
    raise

Prevention

Call quanto freeze() before save_pretrained so dense weights are dropped
Don't merge base-model shards into a quantized export

When it happens

Trigger: Checkpoint that kept the original FP16/BF16 '<prefix>.weight' tensors alongside the quanto-packed '_data'/'_scale' tensors for the same prefix listed in quantization_map.

Common situations: Saving the quantized model without freezing/removing original weights, merging safetensors shards from the base model and the quantized export, or a export tool that writes both representations.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ebae301945ad84f0. Report an issue: GitHub.