sgl-project/sglang · critical · ValueError

Expected len(mlp_layer_types) == num_hidden_layers, got {len

Error message

Expected len(mlp_layer_types) == num_hidden_layers, got {len(mlp_layer_types)} and {cfg.num_hidden_layers}

What it means

Mellum supports per-layer sparse (MoE) vs dense MLPs driven by config.mlp_layer_types, which must list one entry per hidden layer. The model validates len(mlp_layer_types) == num_hidden_layers at init and rejects mismatches. A mismatch means the config is internally inconsistent about layer count.

Source

Thrown at python/sglang/srt/models/mellum.py:405

            max_position_embeddings=max_position_embeddings,
            head_dim=head_dim,
            rms_norm_eps=rms_norm_eps,
            attention_bias=attention_bias,
            config=config,
            quant_config=quant_config,
            prefix=add_prefix("self_attn", prefix),
            sliding_window_size=sliding_window_size,
            alt_stream=alt_stream,
        )

        self.attn_tp_size = get_parallel().attn_tp_size
        self.attn_tp_rank = get_parallel().attn_tp_rank

        mlp_layer_types = cfg.mlp_layer_types
        num_experts = cfg.num_experts

        if len(mlp_layer_types) != cfg.num_hidden_layers:
            raise ValueError(
                "Expected len(mlp_layer_types) == num_hidden_layers, got "
                f"{len(mlp_layer_types)} and {cfg.num_hidden_layers}"
            )

        def _is_sparse(lid: int) -> bool:
            if lid < 0 or lid >= cfg.num_hidden_layers:
                return False
            mlp_type = mlp_layer_types[lid]
            if mlp_type not in ("sparse", "dense"):
                raise ValueError(
                    f"Unsupported mlp_layer_types[{lid}]={mlp_type}; "
                    "expected 'sparse' or 'dense'"
                )
            return mlp_type == "sparse"

        self.is_layer_sparse = _is_sparse(layer_id)

        if self.is_layer_sparse:

View on GitHub (pinned to 0132848349)

Solutions

1. Regenerate mlp_layer_types so it has exactly num_hidden_layers entries (e.g. ["dense","sparse",...])
2. Or correct num_hidden_layers to match the checkpoint's actual layer count
3. Prefer loading the original config.json shipped with the checkpoint rather than merging configs

Example fix

// before
"num_hidden_layers": 28, "mlp_layer_types": ["sparse"] * 27
// after
"num_hidden_layers": 28, "mlp_layer_types": ["dense"] + ["sparse"] * 27

cfg = AutoConfig.from_pretrained(path)
assert len(cfg.mlp_layer_types) == cfg.num_hidden_layers, (
    len(cfg.mlp_layer_types), cfg.num_hidden_layers)

def mlp_types_consistent(cfg) -> bool:
    ml = getattr(cfg, "mlp_layer_types", None)
    return ml is not None and len(ml) == cfg.num_hidden_layers

When it happens

Trigger: config.json has mlp_layer_types of a different length than num_hidden_layers, e.g. after pruning layers, editing num_hidden_layers, or pasting an mlp_layer_types list from another model size.

Common situations: Fine-tuning scripts that override num_hidden_layers; manual config surgery when swapping MoE layers; mismatched config/checkpoint revisions of Mellum.

Related errors

• Unsupported mlp_layer_types[{lid}]={mlp_type}; expected 'spa
• Sparse MLP requested but num_experts <= 0 in Mellum config
• The hpc_ops MoE runner backend does not support fused shared
• The hpc_ops MoE runner backend does not support apply_router
• The hpc_ops MoE runner backend does not support no_combine (

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/731582cf70d243ef. Report an issue: GitHub.