{"record":{"id":"ec3049977a7a3250","repo":"sgl-project/sglang","slug":"a-quantized-component-name-r-checkpoint-requires","errorCode":null,"errorMessage":"A quantized {component_name!r} checkpoint requires an in-tree native encoder; got {model_cls.__name__}","messagePattern":"A quantized (.+?) checkpoint requires an in-tree native encoder; got (.+?)","errorType":"validation","errorClass":"ComponentCheckpointUnsupportedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py","lineNumber":310,"sourceCode":"            )\n        if explicit_quantization not in _ONLINE_ENCODER_QUANTIZATIONS:\n            raise ComponentCheckpointUnsupportedError(\n                f\"Online quantization {explicit_quantization!r} is not supported \"\n                f\"for native encoders; choose one of \"\n                f\"{sorted(_ONLINE_ENCODER_QUANTIZATIONS)}\"\n            )\n        from sglang.multimodal_gen.runtime.layers.quantization import (\n            get_quantization_config,\n        )\n\n        model_config.quant_config = get_quantization_config(explicit_quantization)(\n            ignored_layers=ignored_layers\n        )\n        quant_config = model_config.quant_config\n    if quant_config is None:\n        return\n    if not issubclass(model_cls, EncoderTensorParallelMixin):\n        raise ComponentCheckpointUnsupportedError(\n            f\"A quantized {component_name!r} checkpoint requires an in-tree \"\n            \"native encoder; \"\n            f\"got {model_cls.__name__}\"\n        )\n\n\ndef _resolve_and_configure_encoder_quantization(\n    model_config: EncoderConfig,\n    component_config: dict,\n    component_model_path: str,\n    component_weights_path: str,\n    component_name: str,\n    explicit_quantization: str | None = None,\n    ignored_layers: list[str] | None = None,\n) -> type[nn.Module]:\n    architectures = getattr(model_config, \"architectures\", [])\n    try:\n        model_cls, _ = ModelRegistry.resolve_model_cls(architectures)","sourceCodeStart":292,"sourceCodeEnd":328,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py#L292-L328","documentation":"ComponentCheckpointUnsupportedError raised when a quant_config was resolved (from checkpoint or online override) but the resolved model class does not subclass EncoderTensorParallelMixin. Quantized weight processing (repacking, TP slicing) is only implemented for in-tree native encoders, so external/transformers-delegated architectures are rejected.","triggerScenarios":"Loading a quantized checkpoint (or applying online quantization) for an encoder whose architecture resolves to a transformers model class rather than an in-tree class implementing EncoderTensorParallelMixin — e.g. a quantized CLIP variant that has no native sglang implementation.","commonSituations":"New or niche encoder architectures only supported via transformers fallback; quantized community checkpoints of encoders the runtime only delegates; adding a new encoder without registering a native implementation.","solutions":["Use an unquantized checkpoint for this encoder architecture","Switch to a supported in-tree encoder variant (e.g. the native CLIP implementation) that subclasses EncoderTensorParallelMixin","Contribute/register a native encoder implementation with the mixin to support quantization","Dequantize the checkpoint to FP16/BF16 before loading"],"exampleFix":"# before\nload_customized(model_cls=TransformersCLIPModel, ..., quant_config=fp8_cfg)\n\n# after\nload_customized(model_cls=NativeCLIPModel, ..., quant_config=fp8_cfg)  # in-tree, has mixin","handlingStrategy":"type-guard","validationCode":"from sglang.multimodal_gen.runtime import EncoderTensorParallelMixin\nif quant_config is not None and not issubclass(model_cls, EncoderTensorParallelMixin):\n    quant_config = None  # or fail fast with a clear message","typeGuard":"def quantizable_encoder(model_cls) -> TypeGuard[type]:\n    return issubclass(model_cls, EncoderTensorParallelMixin)","tryCatchPattern":null,"preventionTips":["Register native implementations with EncoderTensorParallelMixin before enabling quantization","Fall back to unquantized weights for delegated architectures"],"tags":["quantization","architecture-unsupported","text-encoder","tensor-parallel"],"backgroundTag":"unsupported-model-architecture","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}