sgl-project/sglang · error · ValueError

Tensor parallel size {self.tp_size} is greater than the numb

Error message

Tensor parallel size {self.tp_size} is greater than the number of experts {config.num_experts}.

What it means

LFM2-MoE's expert layer requires tp_size <= num_experts (lfm2_moe.py:111): with tensor parallelism each rank hosts num_experts/tp_size experts, so more ranks than experts leaves some ranks with none and is rejected at init.

Source

Thrown at python/sglang/srt/models/lfm2_moe.py:111

    - Sigmoid scoring (not softmax) - auxiliary-loss-free style
    - Expert bias (fp32) for load balancing
    - Bias affects selection only, not weighting
    - Uses FusedMoE for efficient batched expert computation
    """

    def __init__(
        self,
        config: Lfm2MoeConfig,
        layer_idx: int,
        quant_config: Optional[QuantizationConfig] = None,
        prefix: str = "",
    ):
        super().__init__()
        self.tp_size = get_parallel().tp_size
        self.routed_scaling_factor = config.routed_scaling_factor

        if self.tp_size > config.num_experts:
            raise ValueError(
                f"Tensor parallel size {self.tp_size} is greater than "
                f"the number of experts {config.num_experts}."
            )

        # Gate (router) - outputs logits for each expert
        self.gate = ReplicatedLinear(
            config.hidden_size,
            config.num_experts,
            bias=False,
            quant_config=None,
            prefix=add_prefix("gate", prefix),
        )

        # Expert bias (fp32) - affects selection but not weighting
        if config.use_expert_bias:
            self.expert_bias = nn.Parameter(
                torch.zeros(config.num_experts, dtype=torch.float32)
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Reduce --tp-size to <= num_experts
2. Prefer expert parallel (--ep-size / --enable-ep-moe) to scale across GPUs
3. Confirm num_experts in the model config before sizing the cluster

Example fix

# before
--tp-size 8   (model has 6 experts)
# after
--tp-size 4 --enable-ep-moe --dp-size 2

assert args.tp_size <= cfg.num_experts

When it happens

Trigger: Launching LFM2 MoE with --tp-size N where N > config.num_experts (e.g. tp=8 with 6 experts).

Common situations: Serving small-expert MoEs on large GPU counts; reusing TP settings from dense models.

Related errors

• TP size {self.tp_size} > num_experts {config.num_experts}.
• The output_size of gate's and up's weight = {intermediate_si
• The input_size of down's weight = {intermediate_size_per_par
• expert-pack v1 supports only single-GPU TP=EP=1
• The output_size of gate's and up's weight = {intermediate_si

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8a4410283bbc3587. Report an issue: GitHub.