{"record":{"id":"2d2b981e4a9d20d1","repo":"sgl-project/sglang","slug":"serialized-quantized-component-weights-cannot-use","errorCode":null,"errorMessage":"Serialized quantized component weights cannot use a stacked parameter-name mapping","messagePattern":"Serialized quantized component weights cannot use a stacked parameter-name mapping","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py","lineNumber":197,"sourceCode":"        srt_quant_config = _get_srt_encoder_quant_config(\n            component_config,\n            model_cls,\n        )\n        if srt_quant_config is not None:\n            return srt_quant_config\n\n    quant_config = get_quant_config(component_config, component_model_path)\n    name_mapper = None\n    parameter_name_mapper = None\n    if model_cls is not None:\n        mapping = vars(model_cls).get(\"param_names_mapping\", {})\n        if mapping:\n            mapping_fn = get_param_names_mapping(mapping)\n\n            def parameter_name_mapper(name: str) -> str:\n                mapped_name, merge_index, _ = mapping_fn(name)\n                if merge_index is not None:\n                    raise ValueError(\n                        \"Serialized quantized component weights cannot use a \"\n                        \"stacked parameter-name mapping\"\n                    )\n                return mapped_name\n\n            def name_mapper(name: str) -> str:\n                # Layer-prefix metadata omits the suffix that many model\n                # mappings use to delimit a parameter name.\n                mapped_name = parameter_name_mapper(f\"{name}.weight\")\n                return mapped_name.removesuffix(\".weight\")\n\n    if names_gguf_checkpoint(component_weights_path):\n        if quant_config is not None:\n            raise ValueError(\n                \"A GGUF encoder checkpoint cannot be combined with a second \"\n                \"quantization declaration\"\n            )\n        tensor_meta = read_gguf_tensor_meta(component_weights_path)","sourceCodeStart":179,"sourceCodeEnd":215,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py#L179-L215","documentation":"When loading serialized quantized component weights, the parameter name mapper forbids stacked parameter mappings: if the configured mapping returns a merge_index for a name (i.e. the name maps into one of several stacked shards), this ValueError is raised, because quantized weights cannot be merged/split after quantization.","triggerScenarios":"A component configured with a stacked parameter-name mapping (used to fuse QKV or MLP weights) combined with a serialized quantized checkpoint; get_param_names_mapping(mapping_fn)(name) returns a non-None merge_index.","commonSituations":"Reusing a fusion-oriented name mapping (written for unquantized fused loading) with a quantized encoder checkpoint; new components importing mappings from fused-attention models; config specifying qkv_merged-style mappings for quantized weights.","solutions":["Use the non-stacked (per-parameter) name mapping for the quantized checkpoint","Load the unquantized checkpoint if stacked/fused loading is required","If you own the mapping, provide a variant that returns merge_index=None for quantized paths"],"exampleFix":null,"handlingStrategy":"type-guard","validationCode":"mapped, merge_idx, _ = mapping_fn(sample_name)\nassert merge_idx is None, \"stacked mapping cannot be used with quantized weights\"","typeGuard":"def is_non_stacked_mapping(mapping_fn, names) -> bool:\n    return all(mapping_fn(n)[1] is None for n in names)","tryCatchPattern":null,"preventionTips":["Keep separate name mappings for quantized (flat) and unquantized (fused) checkpoints","Never reuse QKV-fusion mappings with serialized quantized weights"],"tags":["quantization","state-dict","parameter-mapping","text-encoder"],"backgroundTag":"state-dict-key-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}