{"record":{"id":"619c4bdc0c858d55","repo":"sgl-project/sglang","slug":"missing-modelslim-moe-quantization-description-for","errorCode":null,"errorMessage":"Missing ModelSlim MoE quantization description for layer {prefix}: {joined_status}","messagePattern":"Missing ModelSlim MoE quantization description for layer (.+?): (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/modelslim/modelslim.py","lineNumber":406,"sourceCode":"            # Build a helpful error message listing all attempted key patterns\n            all_attempted = []\n            for candidate in self._quant_prefix_candidates(prefix):\n                for gate_name, up_name, down_name in naming_conventions:\n                    w13_keys = [\n                        f\"{candidate}.0.{gate_name}.weight\",\n                        f\"{candidate}.0.{up_name}.weight\",\n                    ]\n                    w2_key = f\"{candidate}.0.{down_name}.weight\"\n                    w13_found = any(k in self.quant_description for k in w13_keys)\n                    w2_found = w2_key in self.quant_description\n                    status = (\n                        f\"{candidate} \"\n                        f\"({gate_name}/{up_name}=\"\n                        f\"{'found' if w13_found else 'missing'}, \"\n                        f\"{down_name}={'found' if w2_found else 'missing'})\"\n                    )\n                    all_attempted.append(status)\n            raise ValueError(\n                f\"Missing ModelSlim MoE quantization description for layer {prefix}: \"\n                + \"; \".join(all_attempted)\n            )\n\n        # Map scheme names to classes\n        scheme_map = dict(\n            moe_quant_schemes\n        )  # dict: \"W4A4_DYNAMIC\" -> ModelSlimW4A4Int4MoE, etc.\n\n        # Instantiate the schemes\n        def instantiate(name, weight_group):\n            cls = scheme_map.get(name)\n            if cls is None:\n                logger.warning(\n                    f\"Unsupported scheme '{name}' for layer {resolved_prefix}\"\n                )\n                return None\n            return cls(self, weight_group)","sourceCodeStart":388,"sourceCodeEnd":424,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/modelslim/modelslim.py#L388-L424","documentation":"The ModelSlim quant_description contains no (or only partial) entries for the MoE layer's gate/up (w13) and down (w2) projections, after trying all candidate prefixes. The error message lists every attempted prefix and whether each component was found or missing, so it tells you exactly what keys the loader looked for.","triggerScenarios":"get_moe_scheme iterates candidate prefixes and finds w13 or w2 missing (e.g. gate found but up missing, or both missing) for every candidate; then raises with the per-candidate found/missing status joined by ';'.","commonSituations":"MoE experts excluded from quantization in the msModelSlim command (--disable/whitelist filters); prefix shift between the quant tool and SGLang model implementation (extra or missing 'model.' segment); DeepSeek/Qwen-MoE layer names differing from the assumed gate/up/down naming; stale quant config from an older model structure.","solutions":["Read the error's attempted-prefix list and compare against actual keys in the quant_description file","If MoE layers were intentionally skipped, use a quant config/loader path that keeps experts unquantized rather than ModelSlimMoE","Re-quantize including MoE expert weights with the same msModelSlim configuration used for dense layers","If prefixes are shifted (extra/missing 'model.'), fix the prefix resolution or rename keys in the description file"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"def moe_fully_described(qd: dict, prefix: str) -> tuple[bool, str]:\n    missing = [n for n in (\"gate_proj\", \"up_proj\", \"down_proj\")\n               if f\"{prefix}.{n}.weight\" not in qd]\n    return (not missing), f\"missing: {missing} at {prefix}\"\n\nok, detail = moe_fully_described(config.quant_description, moe_prefix)\nassert ok, detail","typeGuard":null,"tryCatchPattern":"try:\n    config.get_quant_method(layer, prefix)\nexcept ValueError as e:\n    if \"Missing ModelSlim MoE quantization description\" in str(e):\n        # parse the attempted-prefix list in the message to see which keys were sought\n        raise","preventionTips":["Pre-flight scan: every MoE prefix in the model must have gate/up/down entries in quant_description","Keep model architecture and quant config in lockstep (same checkpoint revision)","Parse the error's candidate list to diagnose prefix-shift issues quickly"],"tags":["modelslim","quantization","moe","missing-keys","config"],"backgroundTag":"missing-quantization-config-keys","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}