{"record":{"id":"8d5a1759d3915185","repo":"sgl-project/sglang","slug":"found-different-quantization-schemes-for-shard-pr","errorCode":null,"errorMessage":"Found different quantization schemes for {shard_proj_names} in {layer_name}. SGLang requires all to use the same scheme.","messagePattern":"Found different quantization schemes for (.+?) in (.+?)\\. SGLang requires all to use the same scheme\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/compressed_tensors/utils.py","lineNumber":62,"sourceCode":"        shard_names = [\n            layer_name.replace(proj_name, shard_proj_name)\n            for shard_proj_name in shard_proj_names\n        ]\n\n        # Layer should be ignored if shards are ignored.\n        should_ignore_layer = None\n        for shard_name in shard_names:\n            should_ignore_shard = check_equal_or_regex_match(\n                layer_name=shard_name, targets=ignore\n            )\n\n            # If shard_idx=0, set layer ignore to match shard.\n            if should_ignore_layer is None:\n                should_ignore_layer = should_ignore_shard\n\n            # If shard_idx=1+ confirm scheme matches prior shards.\n            elif should_ignore_shard != should_ignore_layer:\n                raise ValueError(\n                    f\"Found different quantization schemes for \"\n                    f\"{shard_proj_names} in {layer_name}. SGLang \"\n                    \"requires all to use the same scheme.\"\n                )\n\n    # Unfused layers like down_proj and o_proj will match\n    # the safetensors checkpoint already.\n    else:\n        should_ignore_layer = check_equal_or_regex_match(\n            layer_name=layer_name, targets=ignore\n        )\n\n    assert should_ignore_layer is not None\n    return should_ignore_layer\n\n\ndef check_equal_or_regex_match(layer_name: str, targets: Iterable[str]) -> bool:\n    \"\"\"","sourceCodeStart":44,"sourceCodeEnd":80,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/compressed_tensors/utils.py#L44-L80","documentation":"When SGLang maps a fused layer (e.g. gate_proj+up_proj → gate_up_proj) onto compressed-tensors targets, all shards of a fused projection must agree on being quantized or not. If shard 0 is ignored (unquantized) but shard 1 is quantized (or vice versa), this ValueError is raised from should_ignore_layer.","triggerScenarios":"A checkpoint where q_proj is quantized but k_proj/v_proj are not (or gate_proj without up_proj) — the fused layer's shards disagree on should_ignore.","commonSituations":"Partial-quantization recipes that quantize attention but not KV projections, then fuse; mixed-precision models from custom finetunes.","solutions":["Re-quantize so all shards of each fused projection share the same scheme (quantize all of q/k/v or none; gate+up together)","If partial quant is intended, disable fusion of those projections or edit targets so they align","Inspect the compressed-tensors config_targets list for missing shard entries"],"exampleFix":"// before: targets cover model.layers.0.self_attn.q_proj only\n// after: include k_proj and v_proj (or remove q_proj) so the fused projection is uniform","handlingStrategy":"validation","validationCode":"targets = set(cfg[\"quantization_config\"].get(\"config_lists\") or [])\n# ensure fused projection shards are uniformly covered\nfor fused in (\"qkv_proj\", \"gate_up_proj\"):\n    shards = {\"q_proj\",\"k_proj\",\"v_proj\"} if fused==\"qkv_proj\" else {\"gate_proj\",\"up_proj\"}\n    covered = [any(s in t for t in targets) for s in shards]\n    assert all(covered) or not any(covered), f\"mixed quantization across {fused} shards\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Quantize projections in fused groups (all of q/k/v, gate+up together)","Use wildcard targets covering whole module groups"],"tags":["quantization","compressed-tensors","fused-layers","config-mismatch"],"backgroundTag":"inconsistent-quantization-targets","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}