sgl-project/sglang · error · ComponentCheckpointUnsupportedError

f"Transformers-managed {component_name!r} quantization requi

Error message

f"Transformers-managed {component_name!r} quantization requires a top-level quantization_config; got metadata from {quant_spec.source!r}"

What it means

For a checkpoint declaring bitsandbytes quantization, the loader requires the metadata to come from the top-level 'quantization_config' field. If the bitsandbytes declaration is discovered via another source (e.g. a nested or alternate key), ComponentCheckpointUnsupportedError is raised because Transformers can only natively manage top-level quantization_config.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/component_loaders/component_loader.py:79

    """A component checkpoint is unsupported and must not use native fallback."""


class NativeComponentLoaderRequired(RuntimeError):
    """The customized loader must defer to the native library loader."""


def uses_native_transformers_bnb4(config: object, component_name: str) -> bool:
    """Validate a serialized BnB4 checkpoint owned by Transformers."""
    try:
        quant_spec = resolve_checkpoint_quant_spec(config)
    except (TypeError, ValueError) as error:
        raise ComponentCheckpointUnsupportedError(
            f"Cannot parse checkpoint quantization for {component_name!r}: {error}"
        ) from error
    if quant_spec is None or quant_spec.declared_method != "bitsandbytes":
        return False
    if quant_spec.source != "quantization_config":
        raise ComponentCheckpointUnsupportedError(
            f"Transformers-managed {component_name!r} quantization requires "
            "a top-level quantization_config; "
            f"got metadata from {quant_spec.source!r}"
        )

    load_in_4bit = quant_spec.config.get(
        "load_in_4bit", quant_spec.config.get("_load_in_4bit")
    )
    load_in_8bit = quant_spec.config.get(
        "load_in_8bit", quant_spec.config.get("_load_in_8bit", False)
    )
    if load_in_4bit is not True or load_in_8bit is True:
        raise ComponentCheckpointUnsupportedError(
            f"Transformers-managed {component_name!r} quantization supports only "
            "serialized BitsAndBytes 4-bit checkpoints"
        )
    return True

View on GitHub (pinned to 0132848349)

Solutions

1. Move the bitsandbytes quantization metadata to the top-level 'quantization_config' key of the component config
2. Re-export the model through a current transformers/diffusers save so quantization_config lands top-level
3. Otherwise use a non-quantized (or fp8) export of the component

Example fix

// before: config.json
{ "transformers_config": { "quantization_config": { "quant_method": "bitsandbytes" } } }
// after
{ "quantization_config": { "quant_method": "bitsandbytes" } }

qc = config.get("quantization_config")
assert qc is None or isinstance(qc, dict), "quantization_config must be a top-level dict"

When it happens

Trigger: A component config where bitsandbytes is declared in a non-top-level location (e.g. inside a nested section or a legacy key), so quant_spec.source != 'quantization_config' while declared_method == 'bitsandbytes'.

Common situations: Checkpoints re-serialized by third-party tools that moved quantization_config into a sub-dict; mixed old/new diffusers exports where bnb metadata lives elsewhere; hand-merged configs.

Related errors

• f"Cannot parse checkpoint quantization for {component_name!r
• {component_name!r} does not support an explicit quantization
• Cannot parse checkpoint quantization metadata for {component
• The quantization method `{quantization}` is already exists.
• The quantization config must be a subclass of `QuantizationC

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8580d668d10de981. Report an issue: GitHub.