sgl-project/sglang · error · ValueError

{component_name!r} does not support an explicit quantization

Error message

{component_name!r} does not support an explicit quantization override; use a self-describing quantized component checkpoint when supported

What it means

ComponentLoader.load rejects an explicit per-component quantization override (server_args.component_quantizations[name]) when the loader does not set supports_online_quantization_override. Such components must instead ship as self-describing pre-quantized checkpoints.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/component_loaders/component_loader.py:238

        component_model_path: str,
        server_args: ServerArgs,
        component_name: str,
        transformers_or_diffusers: str,
    ) -> tuple[AutoModel, float]:
        """
        Template method that standardizes logging around the core load implementation.
        The priority of loading method is:
            1. load customized component
            2. load native diffusers/transformers component
        If all of the above methods failed, an error will be thrown

        """
        component_quantization = server_args.component_quantizations.get(component_name)
        if (
            component_quantization is not None
            and not self.supports_online_quantization_override
        ):
            raise ValueError(
                f"{component_name!r} does not support an explicit quantization "
                "override; "
                "use a self-describing quantized component checkpoint when supported"
            )

        gpu_mem_before_loading = current_platform.get_available_gpu_memory()
        logger.info(
            "Loading %s from %s. avail mem: %.2f GB",
            component_name,
            component_model_path,
            gpu_mem_before_loading,
        )
        attn_backend = None
        component_attn_name = None
        if get_component_attn_backend_context() is None:
            attn_backend, matched_backend_key = (
                server_args.resolve_component_attention_backend(component_name)
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Remove the quantization override for that component from server args / CLI config
2. Use a checkpoint that is already quantized on disk (self-describing quantization_config)
3. If you are the loader author and the component genuinely supports online quantization, set supports_online_quantization_override = True on the loader

Example fix

# before
server_args.component_quantizations['text_encoder'] = 'fp8'
# after
del server_args.component_quantizations['text_encoder']  # use pre-quantized fp8 checkpoint

assert name not in server_args.component_quantizations or loader.supports_online_quantization_override, \
    f"{name} does not accept quantization overrides"

When it happens

Trigger: Launching with a server args entry like component_quantizations={'text_encoder': 'fp8'} for a component whose loader class has supports_online_quantization_override = False (default).

Common situations: Copy-pasting CLI flags for on-the-fly quantization onto a component (e.g. an adapter or bridge) that only accepts serialized quantized weights; enabling quantization globally and having it inherited by unsupported components.

Related errors

• f"Cannot parse checkpoint quantization for {component_name!r
• f"Transformers-managed {component_name!r} quantization requi
• Cannot parse checkpoint quantization metadata for {component
• The quantization method `{quantization}` is already exists.
• The quantization config must be a subclass of `QuantizationC

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7d401fedaa82ae12. Report an issue: GitHub.