sgl-project/sglang · error · ValueError

Tensor parallel size {self.tp_size} is greater than the numb

Error message

Tensor parallel size {self.tp_size} is greater than the number of experts {config.n_routed_experts}.

What it means

The MoE block requires tp_size <= number of routed experts because experts are distributed across TP ranks (at least one expert per rank). Exceeding n_routed_experts makes per-rank expert sharding impossible.

Source

Thrown at python/sglang/srt/models/mimo_v2.py:391

class MiMoV2MoE(nn.Module):

    def __init__(
        self,
        config: MiMoV2Config,
        layer_id: int,
        quant_config: Optional[QuantizationConfig] = None,
        prefix: str = "",
        is_nextn: bool = False,
    ):
        super().__init__()
        self.tp_size = get_parallel().tp_size

        self.config = config
        self.layer_id = layer_id

        if self.tp_size > config.n_routed_experts:
            raise ValueError(
                f"Tensor parallel size {self.tp_size} is greater than "
                f"the number of experts {config.n_routed_experts}."
            )

        if config.hidden_act != "silu":
            raise ValueError(
                f"Unsupported activation: {config.hidden_act}. "
                "Only silu is supported for now."
            )

        self.gate = MoEGate(
            config=config,
            quant_config=quant_config,
            prefix=add_prefix("gate", prefix),
            is_nextn=is_nextn,
        )

        experts_type = get_moe_impl_class(quant_config)

View on GitHub (pinned to 0132848349)

Solutions

1. Lower --tp to at most config.n_routed_experts
2. Use expert parallelism (--ep-size) instead of pure TP to scale across more GPUs
3. Verify n_routed_experts in the config matches the checkpoint

Example fix

# before
python -m sglang.launch_server --tp 256 ...
# after
python -m sglang.launch_server --tp 8 --ep-size 256 ...

tp = get_parallel().tp_size
if tp > config.n_routed_experts:
    raise SystemExit(f'tp={tp} must be <= n_routed_experts={config.n_routed_experts}')

def tp_supported(tp: int, cfg) -> bool:
    return tp <= cfg.n_routed_experts

When it happens

Trigger: Launching MiMo-v2 with --tp greater than config.n_routed_experts, e.g. tp=256 on a model with 128 routed experts.

Common situations: Scaling tensor parallelism too aggressively on small-expert MoE models; misreading config where n_routed_experts is smaller than the intended GPU count.

Related errors

• Tensor parallel size {self.tp_size} is greater than the numb
• Tensor parallel size {self.tp_size} > num_experts {config.nu
• DeepEP is not installed. Please install DeepEP package from
• The output_size of gate's and up's weight = {intermediate_si
• The input_size of down's weight = {intermediate_size_per_par

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3103d8342cff3c9e. Report an issue: GitHub.