sgl-project/sglang · error · ComponentCheckpointUnsupportedError

A quantized {component_name!r} checkpoint requires an in-tre

Error message

A quantized {component_name!r} checkpoint requires an in-tree native encoder; got {model_cls.__name__}

What it means

ComponentCheckpointUnsupportedError raised when a quant_config was resolved (from checkpoint or online override) but the resolved model class does not subclass EncoderTensorParallelMixin. Quantized weight processing (repacking, TP slicing) is only implemented for in-tree native encoders, so external/transformers-delegated architectures are rejected.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py:310

            )
        if explicit_quantization not in _ONLINE_ENCODER_QUANTIZATIONS:
            raise ComponentCheckpointUnsupportedError(
                f"Online quantization {explicit_quantization!r} is not supported "
                f"for native encoders; choose one of "
                f"{sorted(_ONLINE_ENCODER_QUANTIZATIONS)}"
            )
        from sglang.multimodal_gen.runtime.layers.quantization import (
            get_quantization_config,
        )

        model_config.quant_config = get_quantization_config(explicit_quantization)(
            ignored_layers=ignored_layers
        )
        quant_config = model_config.quant_config
    if quant_config is None:
        return
    if not issubclass(model_cls, EncoderTensorParallelMixin):
        raise ComponentCheckpointUnsupportedError(
            f"A quantized {component_name!r} checkpoint requires an in-tree "
            "native encoder; "
            f"got {model_cls.__name__}"
        )


def _resolve_and_configure_encoder_quantization(
    model_config: EncoderConfig,
    component_config: dict,
    component_model_path: str,
    component_weights_path: str,
    component_name: str,
    explicit_quantization: str | None = None,
    ignored_layers: list[str] | None = None,
) -> type[nn.Module]:
    architectures = getattr(model_config, "architectures", [])
    try:
        model_cls, _ = ModelRegistry.resolve_model_cls(architectures)

View on GitHub (pinned to 0132848349)

Solutions

1. Use an unquantized checkpoint for this encoder architecture
2. Switch to a supported in-tree encoder variant (e.g. the native CLIP implementation) that subclasses EncoderTensorParallelMixin
3. Contribute/register a native encoder implementation with the mixin to support quantization
4. Dequantize the checkpoint to FP16/BF16 before loading

Example fix

# before
load_customized(model_cls=TransformersCLIPModel, ..., quant_config=fp8_cfg)

# after
load_customized(model_cls=NativeCLIPModel, ..., quant_config=fp8_cfg)  # in-tree, has mixin

from sglang.multimodal_gen.runtime import EncoderTensorParallelMixin
if quant_config is not None and not issubclass(model_cls, EncoderTensorParallelMixin):
    quant_config = None  # or fail fast with a clear message

def quantizable_encoder(model_cls) -> TypeGuard[type]:
    return issubclass(model_cls, EncoderTensorParallelMixin)

When it happens

Trigger: Loading a quantized checkpoint (or applying online quantization) for an encoder whose architecture resolves to a transformers model class rather than an in-tree class implementing EncoderTensorParallelMixin — e.g. a quantized CLIP variant that has no native sglang implementation.

Common situations: New or niche encoder architectures only supported via transformers fallback; quantized community checkpoints of encoders the runtime only delegates; adding a new encoder without registering a native implementation.

Related errors

• Online quantization for {component_name!r} requires an in-tr
• A quantized {component_name!r} checkpoint requires an in-tre
• bitsandbytes 4-bit TP only supports column-parallel output s
• bitsandbytes 4-bit TP does not support nested quant states.
• bitsandbytes 4-bit TP shard is not aligned to quantization b

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ec3049977a7a3250. Report an issue: GitHub.