{"record":{"id":"36c92cc155b6ba64","repo":"sgl-project/sglang","slug":"no-modelslim-moe-scheme-found-for-layer-prefix","errorCode":null,"errorMessage":"No ModelSlim MoE scheme found for layer {prefix}","messagePattern":"No ModelSlim MoE scheme found for layer (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/modelslim/modelslim.py","lineNumber":294,"sourceCode":"            prefix_in_quant_config = prefix\n            proj_name = prefix.split(\".\")[-1]\n            if proj_name in packed_modules_mapping_subset:\n                prefix_in_quant_config = prefix.replace(\n                    proj_name, packed_modules_mapping_subset[proj_name][0]\n                )\n            prefix_in_quant_config = self._resolve_quant_prefix(prefix_in_quant_config)\n            if self.is_layer_skipped(\n                prefix, packed_modules_mapping_subset\n            ) or self.is_layer_skipped(prefix, self.packed_modules_mapping):\n                return UnquantizedLinearMethod()\n            layer.scheme = self.get_linear_scheme(layer, prefix_in_quant_config)\n            if layer.scheme is None:\n                return UnquantizedLinearMethod()\n            return ModelSlimLinearMethod(self)\n        elif isinstance(layer, FusedMoE):\n            moe_schemes = self.get_moe_scheme(layer, prefix)\n            if moe_schemes is None:\n                raise ValueError(f\"No ModelSlim MoE scheme found for layer {prefix}\")\n            layer.w13_scheme, layer.w2_scheme = moe_schemes\n            layer.w13_kernel, layer.w2_kernel = (\n                layer.w13_scheme.kernel,\n                layer.w2_scheme.kernel,\n            )\n            return ModelSlimFusedMoEMethod(self)\n        return None\n\n    def get_linear_scheme(\n        self, layer: torch.nn.Module, prefix: Optional[str] = None\n    ) -> Optional[ModelSlimLinearScheme]:\n        \"\"\"\n        get_scheme method adjusted for modelslim, taken from\n        python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py\n        \"\"\"\n\n        linear_quant_schemes = [\n            (\"W4A4_DYNAMIC\", ModelSlimW4A4Int4),","sourceCodeStart":276,"sourceCodeEnd":312,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/modelslim/modelslim.py#L276-L312","documentation":"ModelSlim's quantization config could not resolve a MoE scheme for the given FusedMoE layer prefix. get_moe_scheme() returned None, meaning neither the missing-description path nor the unsupported-scheme path applied (e.g. the layer has no scheme entry and the lookup returned None cleanly). It is thrown from get_quant_method when creating a quant method for a FusedMoE layer.","triggerScenarios":"Loading a checkpoint quantized with ModelSlim where the quant_description in the JSON config has no entries for the MoE layer's w13 (gate/up) and w2 (down) projections under any candidate prefix, or the scheme names present do not map to a registered MoE scheme class, causing get_moe_scheme to return None.","commonSituations":"Quantizing only attention/dense layers with ModelSlim but not the MoE experts; mismatched prefix conventions between the quantization tool output and the loader (renamed layers, different prefix roots); using a newer ModelSlim algorithm (e.g. w8a8 variant) unsupported by this SGLang version; running on non-Ascend hardware where the NPU MoE kernels are unavailable.","solutions":["Inspect the ModelSlim quant_description JSON and confirm entries exist for {prefix}.gate_proj.weight, {prefix}.up_proj.weight, and {prefix}.down_proj.weight","Re-quantize the model with ModelSlim including the MoE expert weights, or add them to the quant config","Check the scheme names in quant_description against the scheme_map in get_moe_scheme and upgrade SGLang if the scheme is newer","If MoE layers should stay unquantized, ensure they are excluded so UnquantizedLinearMethod/Fp8 path is used instead of ModelSlimMoE"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"from sglang.srt.layers.quantization.modelslim.modelslim import ModelSlimConfig\n\ndef check_moe_described(cfg: ModelSlimConfig, prefix: str) -> bool:\n    qd = cfg.quant_description\n    for cand in (prefix, prefix.rsplit(\".\", 1)[0]):\n        keys = [f\"{cand}.gate_proj.weight\", f\"{cand}.up_proj.weight\", f\"{cand}.down_proj.weight\"]\n        if all(k in qd for k in keys):\n            return True\n    return False\n\nassert check_moe_described(config, moe_prefix), f\"MoE {moe_prefix} not described\"","typeGuard":null,"tryCatchPattern":"try:\n    method = config.get_quant_method(layer, prefix)\nexcept ValueError as e:\n    if \"No ModelSlim MoE scheme\" in str(e):\n        logger.error(\"MoE layer %s lacks quant description; check msModelSlim output\", prefix)\n    raise","preventionTips":["Validate quant_description covers every FusedMoE prefix before loading the model","Run msModelSlim with MoE layers included when targeting ModelSlimMoE quantization","Keep quant config and checkpoint produced by the same msModelSlim run"],"tags":["modelslim","quantization","moe","ascend-npu","config"],"backgroundTag":"quantization-scheme-not-found","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}