{"record":{"id":"5973b3c51f39940c","repo":"sgl-project/sglang","slug":"fused-module-layer-name-requires-consistent-qu","errorCode":null,"errorMessage":"Fused module '{layer_name}' requires consistent quant config for {sub_names}","messagePattern":"Fused module '(.+?)' requires consistent quant config for (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/auto_round.py","lineNumber":270,"sourceCode":"            if moe_configs:\n                if len(set(moe_configs)) == 1:\n                    return moe_configs[0]\n                raise ValueError(\n                    f\"Fused MoE layer '{layer_name}' requires \"\n                    f\"consistent quant config for all sub-layers\"\n                )\n\n        # 4. Handle fused QKV or other patterns\n        if self.extra_config:\n            for fusion_key, sub_keys in self.packed_modules_mapping.items():\n                if fusion_key in layer_name and layer_name.count(fusion_key) == 1:\n                    sub_names = [\n                        layer_name.replace(fusion_key, sub_key) for sub_key in sub_keys\n                    ]\n                    sub_configs = [get_config(name, quantized) for name in sub_names]\n                    if len(set(sub_configs)) == 1:\n                        return sub_configs[0]\n                    raise ValueError(\n                        f\"Fused module '{layer_name}' requires \"\n                        f\"consistent quant config for {sub_names}\"\n                    )\n\n        # 5. Fallback or try a regular expression match\n        return get_config(layer_name, quantized)\n\n    def check_quantized(self, weight_bits: int) -> bool:\n        return weight_bits < 16\n\n    def check_cpu_support(self, weight_bits: int) -> None:\n        if weight_bits != 4:\n            raise ValueError(\n                \"SGLang's AutoRound CPU inference path currently supports \"\n                \"only 4-bit AWQ/GPTQ checkpoints because it uses the Intel \"\n                f\"AMX INT4 backend, but got {weight_bits}-bit.\"\n            )\n        if not _is_cpu_amx_available:","sourceCodeStart":252,"sourceCodeEnd":288,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/auto_round.py#L252-L288","documentation":"For fused modules like QKV projections, get_layer_config resolves each sub-layer's config (by replacing the fusion key) and requires them all to match. If e.g. q/k/v have different extra_config entries, it raises rather than guessing one config.","triggerScenarios":"An AutoRound extra_config where the sub-layers of a fused module differ — e.g. different entries for ...self_attn.q_proj vs ...k_proj — while the loader asks for the fused qkv_proj layer's config.","commonSituations":"Per-linear tuning scripts (AutoRound/autoscale) writing asymmetric configs, hand-edits, or checkpoints tuned with different bits for k/v projections.","solutions":["Unify the extra_config entries for all sub-layers of the fused module","Delete the per-sub-layer overrides so the fused layer falls back to the default config","Re-run AutoRound calibration with symmetric per-module settings"],"exampleFix":"// before\n{\"...q_proj\": {\"bits\": 4}, \"...k_proj\": {\"bits\": 8}}\n// after\n{\"...self_attn\": {\"bits\": 4}}","handlingStrategy":"validation","validationCode":"FUSIONS = [(\"qkv_proj\", [\"q_proj\", \"k_proj\", \"v_proj\"])]\nfor fused, subs in FUSIONS:\n    vals = {json.dumps(extra_config.get(s), sort_keys=True) for s in subs if s in extra_config}\n    assert len(vals) <= 1, f\"inconsistent config for {fused}\"","typeGuard":null,"tryCatchPattern":"try:\n    cfg.get_layer_config(layer_name)\nexcept ValueError as e:\n    if \"Fused module\" in str(e): unify_sublayer_configs()\n    raise","preventionTips":["Keep calibration symmetric across q/k/v","Lint extra_config for sub-layer divergence before serving"],"tags":["quantization","auto-round","fused-module","qkv"],"backgroundTag":"inconsistent-configuration","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}