sgl-project/sglang · error · ValueError

Missing ModelSlim MoE quantization description for layer {pr

Error message

Missing ModelSlim MoE quantization description for layer {prefix}: {joined_status}

What it means

The ModelSlim quant_description contains no (or only partial) entries for the MoE layer's gate/up (w13) and down (w2) projections, after trying all candidate prefixes. The error message lists every attempted prefix and whether each component was found or missing, so it tells you exactly what keys the loader looked for.

Source

Thrown at python/sglang/srt/layers/quantization/modelslim/modelslim.py:406

            # Build a helpful error message listing all attempted key patterns
            all_attempted = []
            for candidate in self._quant_prefix_candidates(prefix):
                for gate_name, up_name, down_name in naming_conventions:
                    w13_keys = [
                        f"{candidate}.0.{gate_name}.weight",
                        f"{candidate}.0.{up_name}.weight",
                    ]
                    w2_key = f"{candidate}.0.{down_name}.weight"
                    w13_found = any(k in self.quant_description for k in w13_keys)
                    w2_found = w2_key in self.quant_description
                    status = (
                        f"{candidate} "
                        f"({gate_name}/{up_name}="
                        f"{'found' if w13_found else 'missing'}, "
                        f"{down_name}={'found' if w2_found else 'missing'})"
                    )
                    all_attempted.append(status)
            raise ValueError(
                f"Missing ModelSlim MoE quantization description for layer {prefix}: "
                + "; ".join(all_attempted)
            )

        # Map scheme names to classes
        scheme_map = dict(
            moe_quant_schemes
        )  # dict: "W4A4_DYNAMIC" -> ModelSlimW4A4Int4MoE, etc.

        # Instantiate the schemes
        def instantiate(name, weight_group):
            cls = scheme_map.get(name)
            if cls is None:
                logger.warning(
                    f"Unsupported scheme '{name}' for layer {resolved_prefix}"
                )
                return None
            return cls(self, weight_group)

View on GitHub (pinned to 0132848349)

Solutions

1. Read the error's attempted-prefix list and compare against actual keys in the quant_description file
2. If MoE layers were intentionally skipped, use a quant config/loader path that keeps experts unquantized rather than ModelSlimMoE
3. Re-quantize including MoE expert weights with the same msModelSlim configuration used for dense layers
4. If prefixes are shifted (extra/missing 'model.'), fix the prefix resolution or rename keys in the description file

def moe_fully_described(qd: dict, prefix: str) -> tuple[bool, str]:
    missing = [n for n in ("gate_proj", "up_proj", "down_proj")
               if f"{prefix}.{n}.weight" not in qd]
    return (not missing), f"missing: {missing} at {prefix}"

ok, detail = moe_fully_described(config.quant_description, moe_prefix)
assert ok, detail

try:
    config.get_quant_method(layer, prefix)
except ValueError as e:
    if "Missing ModelSlim MoE quantization description" in str(e):
        # parse the attempted-prefix list in the message to see which keys were sought
        raise

When it happens

Trigger: get_moe_scheme iterates candidate prefixes and finds w13 or w2 missing (e.g. gate found but up missing, or both missing) for every candidate; then raises with the per-candidate found/missing status joined by ';'.

Common situations: MoE experts excluded from quantization in the msModelSlim command (--disable/whitelist filters); prefix shift between the quant tool and SGLang model implementation (extra or missing 'model.' segment); DeepSeek/Qwen-MoE layer names differing from the assumed gate/up/down naming; stale quant config from an older model structure.

Related errors

• No ModelSlim MoE scheme found for layer {prefix}
• Mismatched ModelSlim quantization for W13 in layer {prefix}:
• Unsupported ModelSlim MoE schemes for layer {prefix}: W13='{
• Detected some but not all shards of {prefix} are quantized.
• weight_prefix must be 'w13' or 'w2', got '{weight_prefix}'

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/619c4bdc0c858d55. Report an issue: GitHub.