sgl-project/sglang · error · ValueError

The {component_name!r} checkpoint declares quantization, but

Error message

The {component_name!r} checkpoint declares quantization, but the model did not construct any quantized linear layers

What it means

ValueError raised after weight loading when the checkpoint/config declares quantization but the instantiated model built zero quantized linear layers, so process_weights_after_loading never ran. It catches the silent-mismatch case where quant config and model structure disagree, which would otherwise load garbage or skip requantization entirely.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py:392

) -> int:
    processed_layers = 0
    for module in model.modules():
        if not isinstance(module, (LinearBase, SrtLinearBase)):
            continue
        quant_method = module.quant_method
        if quant_method is None or isinstance(
            quant_method,
            (UnquantizedLinearMethod, SrtUnquantizedLinearMethod),
        ):
            continue
        if process_device is None:
            quant_method.process_weights_after_loading(module)
        else:
            with stage_module_for_post_load(module, process_device):
                quant_method.process_weights_after_loading(module)
        processed_layers += 1
    if processed_layers == 0:
        raise ValueError(
            f"The {component_name!r} checkpoint declares quantization, but the "
            "model did not construct any quantized linear layers"
        )
    return processed_layers


def _require_quantized_encoder_layers(
    model: nn.Module,
    component_name: str,
    quant_config: QuantizationConfig | None = None,
) -> None:
    has_quantized_layers = any(
        isinstance(module, (LinearBase, SrtLinearBase))
        and module.quant_method is not None
        and not isinstance(
            module.quant_method,
            (UnquantizedLinearMethod, SrtUnquantizedLinearMethod),
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure the model class builds its linear layers through the quant config's quant_method (e.g. ColumnParallelLinear with quant_config) so at least one quantized layer exists
2. Check ignored_layers patterns — they may be excluding every linear in the encoder
3. Verify the quant method string in the checkpoint matches a registered method that actually constructs quantized layers
4. If unquantized loading is intended, clear quant_config instead of loading with it set

Example fix

# before
self.proj = nn.Linear(in_features, out_features)  # ignores quant_config

# after
self.proj = ColumnParallelLinear(in_features, out_features, quant_config=quant_config)

if model_config.quant_config is not None:
    assert any(
        getattr(m, "quant_method", None) is not None
        for m in model.modules() if isinstance(m, nn.Linear) or hasattr(m, "quant_method")
    ), "no quantized linear layers constructed"

try:
    _process_quantized_encoder_weights(model, ...)
except ValueError as e:
    if "did not construct any quantized linear layers" in str(e):
        # layer construction ignored quant_config; fix model impl or clear quant_config
        raise

When it happens

Trigger: load_model on a component with model_config.quant_config set but whose model class constructs plain nn.Linear / unquantized columns — e.g. quant method misregistered, ignored_layers covering every linear, or a model implementation that ignores the quant config when building layers.

Common situations: Custom encoder implementations that forget to route Linear construction through the quantization method; overly broad ignored_layers; version mismatch where the quant method name in config maps to a method that creates no quantized layers; name remapping (GGUF) hiding linear modules from detection.

Related errors

• The SRT encoder checkpoint adapter supports only serialized
• Serialized quantized component weights cannot use a stacked
• A GGUF encoder checkpoint cannot be combined with a second q
• {component_name!r} manages its own checkpoint quantization a
• Cannot configure checkpoint quantization for {component_name

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3e516cdd0a31f1e5. Report an issue: GitHub.