sgl-project/sglang · error · ValueError

SGLang diffusion currently supports AutoRound auto_gptq chec

Error message

SGLang diffusion currently supports AutoRound auto_gptq checkpoints, but got {srt_config.packing_format!r}.

What it means

AutoRoundConfig.from_config validates the checkpoint's packing format and only accepts GPTQ-style packs; anything else raises. The SGLang diffusion runtime only implements AutoRound's auto_gptq format.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/auto_round.py:45

        return "auto-round"

    @classmethod
    def get_supported_act_dtypes(cls) -> list[torch.dtype]:
        return SRTConfig.get_supported_act_dtypes()

    @classmethod
    def get_min_capability(cls) -> int:
        return SRTConfig.get_min_capability()

    @classmethod
    def get_config_filenames(cls) -> list[str]:
        return SRTConfig.get_config_filenames()

    @classmethod
    def from_config(cls, config: dict) -> "AutoRoundConfig":
        srt_config = SRTConfig.from_config(config)
        if "gptq" not in srt_config.packing_format:
            raise ValueError(
                "SGLang diffusion currently supports AutoRound auto_gptq "
                f"checkpoints, but got {srt_config.packing_format!r}."
            )
        return cls(srt_config)

    def remap_checkpoint_prefixes(self, param_names_mapping: dict) -> None:
        mapping = get_param_names_mapping(param_names_mapping)
        remapped: dict[str, dict] = {}
        for prefix, layer_config in (self.srt_config.extra_config or {}).items():
            target, _, _ = mapping(f"{prefix}.weight")
            target = target.removesuffix(".weight")
            previous = remapped.setdefault(target, layer_config)
            if previous != layer_config:
                raise ValueError(
                    f"AutoRound fused module {target!r} has inconsistent shard configs."
                )

        self.srt_config.extra_config = remapped

View on GitHub (pinned to 0132848349)

Solutions

1. Re-quantize/export the model with AutoRound's auto_gptq packing format
2. Check the checkpoint's quant config to see its actual packing_format value
3. Use a different runtime that supports the packing format you have

srt = SRTConfig.from_config(config)
if "gptq" not in srt.packing_format:
    raise SystemExit("checkpoint is not auto_gptq packed; re-export with AutoRound gptq packing")

When it happens

Trigger: Loading an AutoRound checkpoint whose SRTConfig packing_format does not contain "gptq" (e.g. "awq", "exl2", or a packed float format) via AutoRoundConfig.from_config.

Common situations: Exporting a model with AutoRound using a non-GPTQ packing and trying to serve it here; checkpoints quantized with a newer AutoRound default format.

Related errors

• AutoRound fused module {target!r} has inconsistent shard con
• Parameter {param_name} not found in the model.
• Unsupported quantized embedding marker for {prefix!r}: {mark
• Unsupported quantized linear marker for {prefix!r}
• Unsupported Comfy INT8 format for {prefix!r}: {marker.get('f

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/407d963db99d5ee1. Report an issue: GitHub.