sgl-project/sglang · error · ValueError

Quanto layers collide after parameter mapping at {mapped_pre

Error message

Quanto layers collide after parameter mapping at {mapped_prefix!r}

What it means

After applying the caller-supplied param_name_mapper to each quantization_map prefix, two different source prefixes map to the same target name. Since the returned QuantoInt8Config is keyed by mapped prefixes, collisions would silently merge distinct layers, so they are rejected.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py:186

                raise ValueError(
                    f"Quanto layer {prefix!r} has incompatible scale "
                    f"{scale_slice.get_dtype()} {scale_shape}"
                )
            for scale_name in (names["input"], names["output"]):
                scale = checkpoint.get_slice(scale_name)
                if (
                    scale.get_dtype() not in _FLOAT_DTYPES
                    or tuple(scale.get_shape()) != ()
                ):
                    raise ValueError(
                        f"Quanto auxiliary scale {scale_name!r} must be a float scalar"
                    )

            mapped_prefix = (
                param_name_mapper(prefix) if param_name_mapper is not None else prefix
            )
            if mapped_prefix in mapped_prefixes:
                raise ValueError(
                    f"Quanto layers collide after parameter mapping at {mapped_prefix!r}"
                )
            mapped_prefixes.add(mapped_prefix)

    return QuantoInt8Config(mapped_prefixes)


__all__ = ["QuantoInt8Config", "inspect_quanto_int8_checkpoint"]

View on GitHub (pinned to 0132848349)

Solutions

1. Fix param_name_mapper to be injective: distinct Quanto prefixes must map to distinct names (usually keep the layer index)
2. Log {prefix -> mapped_prefix} pairs before loading and eyeball for duplicates
3. If two prefixes legitimately target one layer, the checkpoint layout is wrong — re-export instead of remapping

Example fix

# before
mapper = lambda p: re.sub(r"layers\.\d+\.", "layers.", p)  # collapses indices
# after
mapper = lambda p: p.replace("encoder.", "model.encoder.")  # injective rename

mapped = [param_name_mapper(p) if param_name_mapper else p for p in quantization_map]
if len(mapped) != len(set(mapped)):
    raise SystemExit('param_name_mapper is not injective on quantized prefixes')

def mapper_is_injective(mapper, prefixes) -> bool:
    outs = [mapper(p) for p in prefixes]
    return len(outs) == len(set(outs))

try:
    cfg = inspect_quanto_int8_checkpoint(ckpt, mapper)
except ValueError as e:
    if 'collide after parameter mapping' in str(e):
        raise SystemExit('Fix mapper to preserve layer indices (injective mapping)')
    raise

When it happens

Trigger: Calling inspect_quanto_int8_checkpoint(ckpt, param_name_mapper=fn) where fn is not injective on the checkpoint's quantized prefixes — e.g. a mapper that strips a distinguishing index or folds layer names together.

Common situations: Adapters renaming encoder layer names for a different model class (e.g. dropping block indices, mapping 'layers.0' and 'layers.10' both to 'layers.0' via faulty regex), or a shared mapper reused across architectures.

Related errors

• QuantoInt8Config must be constructed from safetensors metada
• Quanto checkpoint is missing quantization_map_base64
• Invalid Quanto quantization_map_base64
• Quanto quantization map must be a non-empty object
• Quanto quantization map entries must be named objects

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d9d35837eaf8775c. Report an issue: GitHub.