sgl-project/sglang · error · ComponentCheckpointUnsupportedError

The SRT encoder checkpoint adapter supports only serialized

Error message

The SRT encoder checkpoint adapter supports only serialized 'fp8', got {quant_spec.declared_method!r}

What it means

For text encoders loaded through the SRT adapter path, only serialized fp8 checkpoints are supported. If the checkpoint's quant spec declares any other quant_method (bnb, gptq, awq, unspecified-with-config, etc.), ComponentCheckpointUnsupportedError is raised in _get_srt_encoder_quant_config.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py:158

    component_name: str,
) -> None:
    """Use Transformers when it owns a standard serialized BnB4 checkpoint."""
    if uses_native_transformers_bnb4(component_config, component_name):
        raise NativeComponentLoaderRequired(
            f"{component_name!r} delegates serialized bitsandbytes checkpoint "
            "loading to Transformers"
        )


def _get_srt_encoder_quant_config(
    component_config: dict,
    model_cls: type[EncoderTensorParallelMixin],
) -> SrtFp8Config | None:
    quant_spec = resolve_checkpoint_quant_spec(component_config)
    if quant_spec is None:
        return None
    if quant_spec.declared_method != "fp8":
        raise ComponentCheckpointUnsupportedError(
            "The SRT encoder checkpoint adapter supports only serialized 'fp8', "
            f"got {quant_spec.declared_method!r}"
        )

    config = dict(quant_spec.config)
    config["packed_modules_mapping"] = model_cls.packed_modules_mapping
    return SrtFp8Config.from_config(config)


def _get_encoder_quant_config(
    component_config: dict,
    component_model_path: str,
    component_weights_path: str,
    model_cls: type[nn.Module] | None = None,
):
    if (
        model_cls is not None
        and issubclass(model_cls, EncoderTensorParallelMixin)

View on GitHub (pinned to 0132848349)

Solutions

Use an fp8-serialized text encoder checkpoint (quant_method: 'fp8') or an unquantized one
Remove the quantization override for the text encoder component so it loads unquantized
Re-quantize the encoder to fp8 with the supported toolchain

Defensive patterns

Strategy: validation

Validate before calling

spec = resolve_checkpoint_quant_spec(encoder_config)
assert spec is None or spec.declared_method == 'fp8', \
    f"SRT encoder adapter only supports fp8, got {spec.declared_method}"

Try / catch

except ComponentCheckpointUnsupportedError as e:
    if "supports only serialized 'fp8'" in str(e):
        switch_to_unquantized_encoder_checkpoint()

Prevention

Use fp8 or unquantized text encoder checkpoints with the SRT path
Keep per-component quantization formats consistent with loader support

When it happens

Trigger: Loading a text encoder component whose config declares e.g. quant_method: 'bitsandbytes' while using the SRT encoder loading path with quantization enabled.

Common situations: Mixing quantization formats across components of a multimodal model (bnb text encoder + fp8 DiT); downloading a community-quantized encoder variant; enabling quantization flags that apply to all components.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0823dc45930ce2f3. Report an issue: GitHub.