{"record":{"id":"f7ecd87b822ba932","repo":"sgl-project/sglang","slug":"mismatched-modelslim-quantization-for-w13-in-layer","errorCode":null,"errorMessage":"Mismatched ModelSlim quantization for W13 in layer {prefix}: {w13_entries}","messagePattern":"Mismatched ModelSlim quantization for W13 in layer (.+?): (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/modelslim/modelslim.py","lineNumber":376,"sourceCode":"        resolved_prefix = prefix\n        for candidate in self._quant_prefix_candidates(prefix):\n            for gate_name, up_name, down_name in naming_conventions:\n                w13_keys = [\n                    f\"{candidate}.0.{gate_name}.weight\",\n                    f\"{candidate}.0.{up_name}.weight\",\n                ]\n                w2_key = f\"{candidate}.0.{down_name}.weight\"\n                w13_entries = {\n                    key: self.quant_description[key]\n                    for key in w13_keys\n                    if key in self.quant_description\n                }\n                if w13_entries and w2_key in self.quant_description:\n                    w13_names = list(w13_entries.values())\n                    # For w13, both projections must agree on the scheme\n                    unique_w13 = set(w13_names)\n                    if len(unique_w13) > 1:\n                        raise ValueError(\n                            \"Mismatched ModelSlim quantization for W13 in layer \"\n                            f\"{prefix}: {w13_entries}\"\n                        )\n                    w13_scheme_name = w13_names[0]\n                    w2_scheme_name = self.quant_description[w2_key]\n                    resolved_prefix = candidate\n                    break\n            if w13_scheme_name is not None:\n                break\n\n        if w13_scheme_name is None:\n            # Build a helpful error message listing all attempted key patterns\n            all_attempted = []\n            for candidate in self._quant_prefix_candidates(prefix):\n                for gate_name, up_name, down_name in naming_conventions:\n                    w13_keys = [\n                        f\"{candidate}.0.{gate_name}.weight\",\n                        f\"{candidate}.0.{up_name}.weight\",","sourceCodeStart":358,"sourceCodeEnd":394,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/modelslim/modelslim.py#L358-L394","documentation":"For a fused MoE W13 weight (gate_proj + up_proj stored together), both sub-projections must be quantized with the identical ModelSlim scheme. This error fires when the quant_description maps gate and up to different scheme names (e.g. one W8A8 and one W4A4), which cannot be represented by a single fused w13 scheme.","triggerScenarios":"get_moe_scheme finds w13_entries for gate_proj and up_proj with differing scheme-name values (len(set(names)) > 1) while the w2 key exists — e.g. quant_description[prefix.gate_proj.weight]=='W8A8' but quant_description[prefix.up_proj.weight]=='W4A8'.","commonSituations":"Manually edited or merged ModelSlim quant configs; re-quantizing only some projections of a layer; a quantization tool bug or mixed calibration runs producing inconsistent per-projection dtypes.","solutions":["Open the ModelSlim quant description JSON and locate the entries reported in the error message","Make gate_proj and up_proj use the same scheme (re-run quantization for the whole layer, not per projection)","Verify the checkpoint wasn't assembled from two differently quantized runs","Regenerate the quantization config with a single msModelSlim run covering all MoE projections"],"exampleFix":"// quant_description (JSON) before\n\"...block.0.moe.gate_proj.weight\": \"W8A8\",\n\"...block.0.moe.up_proj.weight\": \"W4A8\"\n// after\n\"...block.0.moe.gate_proj.weight\": \"W8A8\",\n\"...block.0.moe.up_proj.weight\": \"W8A8\"","handlingStrategy":"validation","validationCode":"def w13_schemes_agree(qd: dict, prefix: str):\n    g = qd.get(f\"{prefix}.gate_proj.weight\")\n    u = qd.get(f\"{prefix}.up_proj.weight\")\n    return g is not None and g == u\n\nassert w13_schemes_agree(config.quant_description, moe_prefix), \"gate/up schemes differ\"","typeGuard":null,"tryCatchPattern":"try:\n    config.get_quant_method(layer, prefix)\nexcept ValueError as e:\n    if \"Mismatched ModelSlim quantization for W13\" in str(e):\n        # fix quant_description and re-run\n        raise","preventionTips":["Quantize whole layers in one msModelSlim run, never per projection","Script a sanity check that gate_proj and up_proj scheme strings match for every MoE block","Never hand-merge quant description JSONs from different runs"],"tags":["modelslim","quantization","moe","fused-weights","config-mismatch"],"backgroundTag":"inconsistent-quantization-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}