sgl-project/sglang · critical · ValueError

Unsupported mlp_layer_types[{lid}]={mlp_type}; expected 'spa

Error message

Unsupported mlp_layer_types[{lid}]={mlp_type}; expected 'sparse' or 'dense'

What it means

Each entry of Mellum's config.mlp_layer_types must be exactly 'sparse' or 'dense'; anything else (typos, nulls, other strings) is rejected when the layer's sparsity is resolved during __init__. The error names the offending layer index and value to make the config bug easy to locate.

Source

Thrown at python/sglang/srt/models/mellum.py:415

        self.attn_tp_size = get_parallel().attn_tp_size
        self.attn_tp_rank = get_parallel().attn_tp_rank

        mlp_layer_types = cfg.mlp_layer_types
        num_experts = cfg.num_experts

        if len(mlp_layer_types) != cfg.num_hidden_layers:
            raise ValueError(
                "Expected len(mlp_layer_types) == num_hidden_layers, got "
                f"{len(mlp_layer_types)} and {cfg.num_hidden_layers}"
            )

        def _is_sparse(lid: int) -> bool:
            if lid < 0 or lid >= cfg.num_hidden_layers:
                return False
            mlp_type = mlp_layer_types[lid]
            if mlp_type not in ("sparse", "dense"):
                raise ValueError(
                    f"Unsupported mlp_layer_types[{lid}]={mlp_type}; "
                    "expected 'sparse' or 'dense'"
                )
            return mlp_type == "sparse"

        self.is_layer_sparse = _is_sparse(layer_id)

        if self.is_layer_sparse:
            if num_experts <= 0:
                raise ValueError(
                    "Sparse MLP requested but num_experts <= 0 in Mellum config"
                )
            self.mlp = Qwen3MoeSparseMoeBlock(
                layer_id=layer_id,
                config=cfg,
                quant_config=quant_config,
                prefix=add_prefix("mlp", prefix),
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect config.json mlp_layer_types[lid] and normalize every entry to exactly 'sparse' or 'dense'
2. Check for typos, casing, or trailing whitespace in the list
3. Validate the list programmatically before loading the model (see validation snippet)

Example fix

// before
"mlp_layer_types": ["Dense", "moe"]
// after
"mlp_layer_types": ["dense", "sparse"]

cfg = AutoConfig.from_pretrained(path)
assert all(v in ("sparse", "dense") for v in cfg.mlp_layer_types), [
    i for i, v in enumerate(cfg.mlp_layer_types) if v not in ("sparse", "dense")]

def mlp_types_valid(cfg) -> bool:
    return all(v in ("sparse", "dense") for v in getattr(cfg, "mlp_layer_types", []))

When it happens

Trigger: mlp_layer_types contains a value like 'moe', 'Sparse', '', or None at index lid; case-mismatched or whitespace-padded strings from manual config edits.

Common situations: Hand-edited config.json during conversion; configs authored from memory instead of copying the original; locale/case differences in generated configs.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Expected len(mlp_layer_types) == num_hidden_layers, got {len
• Sparse MLP requested but num_experts <= 0 in Mellum config
• The hpc_ops MoE runner backend does not support fused shared
• The hpc_ops MoE runner backend does not support apply_router
• The hpc_ops MoE runner backend does not support no_combine (

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3cf8bdcbc3a41c33. Report an issue: GitHub.