sgl-project/sglang · error · ValueError

TP size {self.tp_size} > num_experts {config.num_experts}.

Error message

TP size {self.tp_size} > num_experts {config.num_experts}.

What it means

Laguna's MoE layer shards experts across tensor-parallel ranks; laguna.py:152 enforces tp_size <= num_experts (each rank must get at least one expert). Exceeding it raises at model init because expert sharding would produce empty ranks.

Source

Thrown at python/sglang/srt/models/laguna.py:152

class LagunaMoE(nn.Module):
    def __init__(
        self,
        config: LagunaConfig,
        layer_id: int,
        quant_config: Optional[QuantizationConfig] = None,
        prefix: str = "",
    ):
        super().__init__()
        self.tp_size = get_parallel().tp_size
        self.routed_scaling_factor = config.moe_routed_scaling_factor
        self.router_logit_softcapping = getattr(
            config, "moe_router_logit_softcapping", 0.0
        )

        if self.tp_size > config.num_experts:
            raise ValueError(
                f"TP size {self.tp_size} > num_experts {config.num_experts}."
            )

        self.gate = LagunaMoEGate(config, prefix=add_prefix("gate", prefix))

        self.experts = get_moe_impl_class(quant_config)(
            num_experts=config.num_experts + get_exec().moe.ep_num_redundant_experts,
            top_k=config.num_experts_per_tok,
            layer_id=layer_id,
            hidden_size=config.hidden_size,
            intermediate_size=config.moe_intermediate_size,
            quant_config=quant_config,
            reduce_results=False,
            apply_router_weight_on_input=bool(config.moe_apply_router_weight_on_input),
            prefix=add_prefix("experts", prefix),
        )

        self.topk = TopK(

View on GitHub (pinned to 0132848349)

Solutions

1. Choose tp_size <= num_experts (prefer expert parallelism: --enable-ep-moe / --dp-size)
2. Check config.num_experts before selecting --tp-size
3. Rebalance: use DP + EP for many-GPU serving of small-expert MoEs

Example fix

# before
--tp-size 8 --enable-ep-moe  (model has 4 experts)
# after
--tp-size 4 --enable-ep-moe --dp-size 2

assert args.tp_size <= cfg.num_experts, f"tp {args.tp_size} > experts {cfg.num_experts}"

When it happens

Trigger: Launching with --tp-size larger than config.num_experts, e.g. tp=8 on a model with 4 routed experts.

Common situations: Small-expert-count MoE checkpoints run on many GPUs; copying TP flags from a dense-model deployment.

Related errors

• Tensor parallel size {self.tp_size} is greater than the numb
• The output_size of gate's and up's weight = {intermediate_si
• The input_size of down's weight = {intermediate_size_per_par
• expert-pack v1 supports only single-GPU TP=EP=1
• The output_size of gate's and up's weight = {intermediate_si

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3a75d3561b64c5d1. Report an issue: GitHub.