sgl-project/sglang · error · ValueError

AutoRound fused module {target!r} has inconsistent shard con

Error message

AutoRound fused module {target!r} has inconsistent shard configs.

What it means

When remapping checkpoint prefixes, fused modules (multiple prefixes mapping to one target) must carry identical layer configs in extra_config. If two prefixes that fuse into the same target have different configs, the fusion is ambiguous and this error is raised.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/auto_round.py:59

    @classmethod
    def from_config(cls, config: dict) -> "AutoRoundConfig":
        srt_config = SRTConfig.from_config(config)
        if "gptq" not in srt_config.packing_format:
            raise ValueError(
                "SGLang diffusion currently supports AutoRound auto_gptq "
                f"checkpoints, but got {srt_config.packing_format!r}."
            )
        return cls(srt_config)

    def remap_checkpoint_prefixes(self, param_names_mapping: dict) -> None:
        mapping = get_param_names_mapping(param_names_mapping)
        remapped: dict[str, dict] = {}
        for prefix, layer_config in (self.srt_config.extra_config or {}).items():
            target, _, _ = mapping(f"{prefix}.weight")
            target = target.removesuffix(".weight")
            previous = remapped.setdefault(target, layer_config)
            if previous != layer_config:
                raise ValueError(
                    f"AutoRound fused module {target!r} has inconsistent shard configs."
                )

        self.srt_config.extra_config = remapped
        self.srt_config.block_name_to_quantize = None
        self.srt_config.packed_modules_mapping = self.packed_modules_mapping

    def get_quant_method(self, layer: torch.nn.Module, prefix: str):
        if not isinstance(layer, LinearBase):
            return None

        weight_bits, _, _ = self.srt_config.get_layer_config(layer, prefix)
        if not self.srt_config.check_quantized(weight_bits):
            return UnquantizedLinearMethod()

        return self.srt_config.apply_gptq_quant_layer(
            layer,
            prefix,

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect srt_config.extra_config for the offending prefix pair and make their configs identical
2. Re-export the checkpoint with consistent fused-layer settings
3. Report upstream if a stock AutoRound export triggers this

targets = {}
for prefix, layer_cfg in extra_config.items():
    t = map_prefix(prefix)
    if t in targets and targets[t] != layer_cfg:
        raise SystemExit(f"inconsistent shard configs for fused target {t}")
    targets[t] = layer_cfg

When it happens

Trigger: An AutoRound checkpoint where extra_config assigns different shard configs to prefixes that both map to the same fused target module after weight-name mapping.

Common situations: Hand-edited quant config json; checkpoints produced by an AutoRound version that emits per-shard differing configs for fused layers.

Related errors

• SGLang diffusion currently supports AutoRound auto_gptq chec
• Parameter {param_name} not found in the model.
• Unsupported quantized embedding marker for {prefix!r}: {mark
• Unsupported quantized linear marker for {prefix!r}
• Unsupported Comfy INT8 format for {prefix!r}: {marker.get('f

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/570f49088ba31789. Report an issue: GitHub.