{"record":{"id":"16e51bb9e952f5f1","repo":"sgl-project/sglang","slug":"fused-moe-layer-layer-name-requires-consistent","errorCode":null,"errorMessage":"Fused MoE layer '{layer_name}' requires consistent quant config for all sub-layers","messagePattern":"Fused MoE layer '(.+?)' requires consistent quant config for all sub-layers","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/auto_round.py","lineNumber":255,"sourceCode":"\n        # 2. Determine whether layer should be quantized\n        quantized = not isinstance(layer, ParallelLMHead)\n        if self.block_name_to_quantize:\n            quantized = any(\n                layer_name.startswith(name) for name in self.block_name_to_quantize\n            )\n\n        # 3. Handle fused MoE\n        if self.extra_config and \"fusedmoe\" in layer.__class__.__name__.lower():\n            moe_configs = [\n                get_config(name, quantized)\n                for name in self.extra_config\n                if name.startswith(layer_name)\n            ]\n            if moe_configs:\n                if len(set(moe_configs)) == 1:\n                    return moe_configs[0]\n                raise ValueError(\n                    f\"Fused MoE layer '{layer_name}' requires \"\n                    f\"consistent quant config for all sub-layers\"\n                )\n\n        # 4. Handle fused QKV or other patterns\n        if self.extra_config:\n            for fusion_key, sub_keys in self.packed_modules_mapping.items():\n                if fusion_key in layer_name and layer_name.count(fusion_key) == 1:\n                    sub_names = [\n                        layer_name.replace(fusion_key, sub_key) for sub_key in sub_keys\n                    ]\n                    sub_configs = [get_config(name, quantized) for name in sub_names]\n                    if len(set(sub_configs)) == 1:\n                        return sub_configs[0]\n                    raise ValueError(\n                        f\"Fused module '{layer_name}' requires \"\n                        f\"consistent quant config for {sub_names}\"\n                    )","sourceCodeStart":237,"sourceCodeEnd":273,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/auto_round.py#L237-L273","documentation":"AutoRoundConfig.get_layer_config detects that the queried layer is a fused MoE layer whose sub-layers (experts) have differing per-layer quant configs in extra_config. A fused MoE kernel needs one uniform config, so inconsistency raises.","triggerScenarios":"An AutoRound checkpoint whose quant_config.json / extra_config specifies different quant settings for different experts under the same MoE layer prefix (e.g. some experts 4-bit, some 8-bit), then calling get_quant_method on that layer.","commonSituations":"Mixed-precision expert quantization exports, hand-edited extra_config, or partial per-layer overrides that unintentionally split across expert indices.","solutions":["Make the extra_config entries identical for all sub-layers of the fused MoE layer (remove per-expert overrides)","Re-export the model with uniform quantization for the MoE block","If mixed-precision experts are required, use a path that supports per-expert methods instead of fused MoE"],"exampleFix":"// before (extra_config)\n{\"model.layers.0.mlp.experts.0\": {...4bit...}, \"model.layers.0.mlp.experts.1\": {...8bit...}}\n// after\n{\"model.layers.0.mlp\": {...4bit uniform...}}","handlingStrategy":"validation","validationCode":"import collections\ngroups = collections.defaultdict(set)\nfor name, c in extra_config.items():\n    prefix = name.rsplit(\".experts.\", 1)[0]\n    groups[prefix].add(json.dumps(c, sort_keys=True))\nassert all(len(v) == 1 for v in groups.values()), \"MoE sub-layer configs differ\"","typeGuard":null,"tryCatchPattern":"try:\n    cfg.get_layer_config(layer_name)\nexcept ValueError as e:\n    if \"Fused MoE layer\" in str(e): unify_expert_configs_and_retry()\n    raise","preventionTips":["Never write per-expert quant overrides for fused MoE","Validate extra_config uniformity in a pre-launch check script"],"tags":["quantization","auto-round","moe","fused-layer"],"backgroundTag":"inconsistent-configuration","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}