sgl-project/sglang · error · ValueError

The checkpoint declares quantization, but the model did not…

Error message

The {component_name!r} checkpoint declares quantization, but the model did not construct any quantized linear layers

What it means

ValueError raised after weight loading when the checkpoint/config declares quantization but the instantiated model built zero quantized linear layers, so process_weights_after_loading never ran. It catches the silent-mismatch case where quant config and model structure disagree, which would otherwise load garbage or skip requantization entirely.

Solutions

  1. Ensure the model class builds its linear layers through the quant config's quant_method (e.g. ColumnParallelLinear with quant_config) so at least one quantized layer exists
  2. Check ignored_layers patterns — they may be excluding every linear in the encoder
  3. Verify the quant method string in the checkpoint matches a registered method that actually constructs quantized layers
  4. If unquantized loading is intended, clear quant_config instead of loading with it set

Example fix

# before
self.proj = nn.Linear(in_features, out_features)  # ignores quant_config

# after
self.proj = ColumnParallelLinear(in_features, out_features, quant_config=quant_config)
Defensive patterns

Strategy: try-catch

Validate before calling

if model_config.quant_config is not None:
    assert any(
        getattr(m, "quant_method", None) is not None
        for m in model.modules() if isinstance(m, nn.Linear) or hasattr(m, "quant_method")
    ), "no quantized linear layers constructed"

Try / catch

try:
    _process_quantized_encoder_weights(model, ...)
except ValueError as e:
    if "did not construct any quantized linear layers" in str(e):
        # layer construction ignored quant_config; fix model impl or clear quant_config
        raise

Prevention

When it happens

Trigger: load_model on a component with model_config.quant_config set but whose model class constructs plain nn.Linear / unquantized columns — e.g. quant method misregistered, ignored_layers covering every linear, or a model implementation that ignores the quant config when building layers.

Common situations: Custom encoder implementations that forget to route Linear construction through the quantization method; overly broad ignored_layers; version mismatch where the quant method name in config maps to a method that creates no quantized layers; name remapping (GGUF) hiding linear modules from detection.

Related errors


AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3e516cdd0a31f1e5. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py:392

) -> int:
    processed_layers = 0
    for module in model.modules():
        if not isinstance(module, (LinearBase, SrtLinearBase)):
            continue
        quant_method = module.quant_method
        if quant_method is None or isinstance(
            quant_method,
            (UnquantizedLinearMethod, SrtUnquantizedLinearMethod),
        ):
            continue
        if process_device is None:
            quant_method.process_weights_after_loading(module)
        else:
            with stage_module_for_post_load(module, process_device):
                quant_method.process_weights_after_loading(module)
        processed_layers += 1
    if processed_layers == 0:
        raise ValueError(
            f"The {component_name!r} checkpoint declares quantization, but the "
            "model did not construct any quantized linear layers"
        )
    return processed_layers


def _require_quantized_encoder_layers(
    model: nn.Module,
    component_name: str,
    quant_config: QuantizationConfig | None = None,
) -> None:
    has_quantized_layers = any(
        isinstance(module, (LinearBase, SrtLinearBase))
        and module.quant_method is not None
        and not isinstance(
            module.quant_method,
            (UnquantizedLinearMethod, SrtUnquantizedLinearMethod),
        )

View on GitHub (pinned to 0132848349)