sgl-project/sglang · error · ValueError

Serialized quantized component weights cannot use a stacked

Error message

Serialized quantized component weights cannot use a stacked parameter-name mapping

What it means

When loading serialized quantized component weights, the parameter name mapper forbids stacked parameter mappings: if the configured mapping returns a merge_index for a name (i.e. the name maps into one of several stacked shards), this ValueError is raised, because quantized weights cannot be merged/split after quantization.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py:197

        srt_quant_config = _get_srt_encoder_quant_config(
            component_config,
            model_cls,
        )
        if srt_quant_config is not None:
            return srt_quant_config

    quant_config = get_quant_config(component_config, component_model_path)
    name_mapper = None
    parameter_name_mapper = None
    if model_cls is not None:
        mapping = vars(model_cls).get("param_names_mapping", {})
        if mapping:
            mapping_fn = get_param_names_mapping(mapping)

            def parameter_name_mapper(name: str) -> str:
                mapped_name, merge_index, _ = mapping_fn(name)
                if merge_index is not None:
                    raise ValueError(
                        "Serialized quantized component weights cannot use a "
                        "stacked parameter-name mapping"
                    )
                return mapped_name

            def name_mapper(name: str) -> str:
                # Layer-prefix metadata omits the suffix that many model
                # mappings use to delimit a parameter name.
                mapped_name = parameter_name_mapper(f"{name}.weight")
                return mapped_name.removesuffix(".weight")

    if names_gguf_checkpoint(component_weights_path):
        if quant_config is not None:
            raise ValueError(
                "A GGUF encoder checkpoint cannot be combined with a second "
                "quantization declaration"
            )
        tensor_meta = read_gguf_tensor_meta(component_weights_path)

View on GitHub (pinned to 0132848349)

Solutions

1. Use the non-stacked (per-parameter) name mapping for the quantized checkpoint
2. Load the unquantized checkpoint if stacked/fused loading is required
3. If you own the mapping, provide a variant that returns merge_index=None for quantized paths

mapped, merge_idx, _ = mapping_fn(sample_name)
assert merge_idx is None, "stacked mapping cannot be used with quantized weights"

def is_non_stacked_mapping(mapping_fn, names) -> bool:
    return all(mapping_fn(n)[1] is None for n in names)

When it happens

Trigger: A component configured with a stacked parameter-name mapping (used to fuse QKV or MLP weights) combined with a serialized quantized checkpoint; get_param_names_mapping(mapping_fn)(name) returns a non-None merge_index.

Common situations: Reusing a fusion-oriented name mapping (written for unquantized fused loading) with a quantized encoder checkpoint; new components importing mappings from fused-attention models; config specifying qkv_merged-style mappings for quantized weights.

Related errors

• Parameter {param_name} not found in the model.
• {component_name!r} checkpoint declares quantization metadata
• The SRT encoder checkpoint adapter supports only serialized
• A GGUF encoder checkpoint cannot be combined with a second q
• {component_name!r} manages its own checkpoint quantization a

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2d2b981e4a9d20d1. Report an issue: GitHub.