sgl-project/sglang · error · ValueError

No ModelSlim MoE scheme found for layer {prefix}

Error message

No ModelSlim MoE scheme found for layer {prefix}

What it means

ModelSlim's quantization config could not resolve a MoE scheme for the given FusedMoE layer prefix. get_moe_scheme() returned None, meaning neither the missing-description path nor the unsupported-scheme path applied (e.g. the layer has no scheme entry and the lookup returned None cleanly). It is thrown from get_quant_method when creating a quant method for a FusedMoE layer.

Source

Thrown at python/sglang/srt/layers/quantization/modelslim/modelslim.py:294

            prefix_in_quant_config = prefix
            proj_name = prefix.split(".")[-1]
            if proj_name in packed_modules_mapping_subset:
                prefix_in_quant_config = prefix.replace(
                    proj_name, packed_modules_mapping_subset[proj_name][0]
                )
            prefix_in_quant_config = self._resolve_quant_prefix(prefix_in_quant_config)
            if self.is_layer_skipped(
                prefix, packed_modules_mapping_subset
            ) or self.is_layer_skipped(prefix, self.packed_modules_mapping):
                return UnquantizedLinearMethod()
            layer.scheme = self.get_linear_scheme(layer, prefix_in_quant_config)
            if layer.scheme is None:
                return UnquantizedLinearMethod()
            return ModelSlimLinearMethod(self)
        elif isinstance(layer, FusedMoE):
            moe_schemes = self.get_moe_scheme(layer, prefix)
            if moe_schemes is None:
                raise ValueError(f"No ModelSlim MoE scheme found for layer {prefix}")
            layer.w13_scheme, layer.w2_scheme = moe_schemes
            layer.w13_kernel, layer.w2_kernel = (
                layer.w13_scheme.kernel,
                layer.w2_scheme.kernel,
            )
            return ModelSlimFusedMoEMethod(self)
        return None

    def get_linear_scheme(
        self, layer: torch.nn.Module, prefix: Optional[str] = None
    ) -> Optional[ModelSlimLinearScheme]:
        """
        get_scheme method adjusted for modelslim, taken from
        python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py
        """

        linear_quant_schemes = [
            ("W4A4_DYNAMIC", ModelSlimW4A4Int4),

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect the ModelSlim quant_description JSON and confirm entries exist for {prefix}.gate_proj.weight, {prefix}.up_proj.weight, and {prefix}.down_proj.weight
2. Re-quantize the model with ModelSlim including the MoE expert weights, or add them to the quant config
3. Check the scheme names in quant_description against the scheme_map in get_moe_scheme and upgrade SGLang if the scheme is newer
4. If MoE layers should stay unquantized, ensure they are excluded so UnquantizedLinearMethod/Fp8 path is used instead of ModelSlimMoE

from sglang.srt.layers.quantization.modelslim.modelslim import ModelSlimConfig

def check_moe_described(cfg: ModelSlimConfig, prefix: str) -> bool:
    qd = cfg.quant_description
    for cand in (prefix, prefix.rsplit(".", 1)[0]):
        keys = [f"{cand}.gate_proj.weight", f"{cand}.up_proj.weight", f"{cand}.down_proj.weight"]
        if all(k in qd for k in keys):
            return True
    return False

assert check_moe_described(config, moe_prefix), f"MoE {moe_prefix} not described"

try:
    method = config.get_quant_method(layer, prefix)
except ValueError as e:
    if "No ModelSlim MoE scheme" in str(e):
        logger.error("MoE layer %s lacks quant description; check msModelSlim output", prefix)
    raise

When it happens

Trigger: Loading a checkpoint quantized with ModelSlim where the quant_description in the JSON config has no entries for the MoE layer's w13 (gate/up) and w2 (down) projections under any candidate prefix, or the scheme names present do not map to a registered MoE scheme class, causing get_moe_scheme to return None.

Common situations: Quantizing only attention/dense layers with ModelSlim but not the MoE experts; mismatched prefix conventions between the quantization tool output and the loader (renamed layers, different prefix roots); using a newer ModelSlim algorithm (e.g. w8a8 variant) unsupported by this SGLang version; running on non-Ascend hardware where the NPU MoE kernels are unavailable.

Related errors

• Missing ModelSlim MoE quantization description for layer {pr
• Mismatched ModelSlim quantization for W13 in layer {prefix}:
• Unsupported ModelSlim MoE schemes for layer {prefix}: W13='{
• Detected some but not all shards of {prefix} are quantized.
• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/36c92cc155b6ba64. Report an issue: GitHub.