sgl-project/sglang · critical · ValueError

Tensor parallel size {self.tp_size} is greater than the numb

Error message

Tensor parallel size {self.tp_size} is greater than the number of experts {self.n_routed_experts}.

What it means

AFMoE shards experts across tensor-parallel ranks, so you cannot have more TP ranks than experts. The check fails when tp_size > num_experts because some rank would own zero experts.

Source

Thrown at python/sglang/srt/models/afmoe.py:170

        return topk_weights.to(torch.float32), topk_ids.to(torch.int32)

    def __init__(
        self,
        config: PretrainedConfig,
        quant_config: Optional[QuantizationConfig] = None,
        prefix: str = "",
    ):
        super().__init__()
        self.config = config
        self.rank = get_parallel().tp_rank
        self.tp_size = get_parallel().tp_size

        self.n_routed_experts = getattr(config, "num_experts", None)
        if self.n_routed_experts is None:
            raise ValueError("AfmoeConfig must define `num_experts`.")
        self.top_k = config.num_experts_per_tok
        if self.tp_size > self.n_routed_experts:
            raise ValueError(
                f"Tensor parallel size {self.tp_size} is greater than "
                f"the number of experts {self.n_routed_experts}."
            )

        self.score_func = getattr(config, "score_func", "softmax")
        self.route_norm = getattr(config, "route_norm", True)
        self.route_scale = float(getattr(config, "route_scale", 1.0))
        self.n_group = getattr(config, "n_group", 1)
        self.topk_group = getattr(config, "topk_group", 1)
        self.use_grouped_topk = self.n_group is not None and self.n_group > 1
        self.num_shared_experts = getattr(config, "num_shared_experts", 0)

        self.gate = ReplicatedLinear(
            config.hidden_size,
            self.n_routed_experts,
            bias=False,
            quant_config=None,
            prefix=add_prefix("gate", prefix),

View on GitHub (pinned to 0132848349)

Solutions

1. Reduce --tensor-parallel-size to <= num_experts (e.g. tp=2 for 4 experts)
2. Increase the model's num_experts by using a different checkpoint if more parallelism is required

Example fix

# before
python -m sglang.launch_server --model afmoe-x --tp 8
# after
python -m sglang.launch_server --model afmoe-x --tp 2

assert tp_size <= config.num_experts, f"tp {tp_size} > experts {config.num_experts}"

When it happens

Trigger: Launching with --tp 8 on an AFmoe model whose config has fewer than 8 num_experts; combining large TP with a small expert count.

Common situations: Reusing a high-TP launch command from a dense model on a small MoE model; misreading num_experts_per_tok as num_experts.

Related errors

• num_heads ({self.num_heads}) must be divisible by tp_size ({
• MiniMax H3 attention heads must be divisible by TP size: {ar
• The output_size of gate's and up's weight = {intermediate_si
• The input_size of down's weight = {intermediate_size_per_par
• expert-pack v1 supports only single-GPU TP=EP=1

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/92c1141b2560b478. Report an issue: GitHub.