{"record":{"id":"3e516cdd0a31f1e5","repo":"sgl-project/sglang","slug":"the-component-name-r-checkpoint-declares-quantiz","errorCode":null,"errorMessage":"The {component_name!r} checkpoint declares quantization, but the model did not construct any quantized linear layers","messagePattern":"The (.+?) checkpoint declares quantization, but the model did not construct any quantized linear layers","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py","lineNumber":392,"sourceCode":") -> int:\n    processed_layers = 0\n    for module in model.modules():\n        if not isinstance(module, (LinearBase, SrtLinearBase)):\n            continue\n        quant_method = module.quant_method\n        if quant_method is None or isinstance(\n            quant_method,\n            (UnquantizedLinearMethod, SrtUnquantizedLinearMethod),\n        ):\n            continue\n        if process_device is None:\n            quant_method.process_weights_after_loading(module)\n        else:\n            with stage_module_for_post_load(module, process_device):\n                quant_method.process_weights_after_loading(module)\n        processed_layers += 1\n    if processed_layers == 0:\n        raise ValueError(\n            f\"The {component_name!r} checkpoint declares quantization, but the \"\n            \"model did not construct any quantized linear layers\"\n        )\n    return processed_layers\n\n\ndef _require_quantized_encoder_layers(\n    model: nn.Module,\n    component_name: str,\n    quant_config: QuantizationConfig | None = None,\n) -> None:\n    has_quantized_layers = any(\n        isinstance(module, (LinearBase, SrtLinearBase))\n        and module.quant_method is not None\n        and not isinstance(\n            module.quant_method,\n            (UnquantizedLinearMethod, SrtUnquantizedLinearMethod),\n        )","sourceCodeStart":374,"sourceCodeEnd":410,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py#L374-L410","documentation":"ValueError raised after weight loading when the checkpoint/config declares quantization but the instantiated model built zero quantized linear layers, so process_weights_after_loading never ran. It catches the silent-mismatch case where quant config and model structure disagree, which would otherwise load garbage or skip requantization entirely.","triggerScenarios":"load_model on a component with model_config.quant_config set but whose model class constructs plain nn.Linear / unquantized columns — e.g. quant method misregistered, ignored_layers covering every linear, or a model implementation that ignores the quant config when building layers.","commonSituations":"Custom encoder implementations that forget to route Linear construction through the quantization method; overly broad ignored_layers; version mismatch where the quant method name in config maps to a method that creates no quantized layers; name remapping (GGUF) hiding linear modules from detection.","solutions":["Ensure the model class builds its linear layers through the quant config's quant_method (e.g. ColumnParallelLinear with quant_config) so at least one quantized layer exists","Check ignored_layers patterns — they may be excluding every linear in the encoder","Verify the quant method string in the checkpoint matches a registered method that actually constructs quantized layers","If unquantized loading is intended, clear quant_config instead of loading with it set"],"exampleFix":"# before\nself.proj = nn.Linear(in_features, out_features)  # ignores quant_config\n\n# after\nself.proj = ColumnParallelLinear(in_features, out_features, quant_config=quant_config)","handlingStrategy":"try-catch","validationCode":"if model_config.quant_config is not None:\n    assert any(\n        getattr(m, \"quant_method\", None) is not None\n        for m in model.modules() if isinstance(m, nn.Linear) or hasattr(m, \"quant_method\")\n    ), \"no quantized linear layers constructed\"","typeGuard":null,"tryCatchPattern":"try:\n    _process_quantized_encoder_weights(model, ...)\nexcept ValueError as e:\n    if \"did not construct any quantized linear layers\" in str(e):\n        # layer construction ignored quant_config; fix model impl or clear quant_config\n        raise\n","preventionTips":["Build encoder linears via the quant method, not raw nn.Linear","Keep ignored_layers patterns narrow and test them","Add a unit test asserting at least one quantized layer exists when quant_config is set"],"tags":["quantization","model-mismatch","linear-layers","text-encoder","silent-failure-guard"],"backgroundTag":"quantized-layers-not-constructed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}