sgl-project/sglang · error · ValueError

Tensor parallel size {self.tp_size} is greater than the numb

Error message

Tensor parallel size {self.tp_size} is greater than the number of experts {config.num_experts}.

What it means

Qwen3 MoE requires moe_tp_size <= config.num_experts because expert-parallel/MoE-TP sharding assigns each TP rank at least one expert; with more ranks than experts some rank would own zero experts.

Source

Thrown at python/sglang/srt/models/qwen3_moe.py:241

    # # )
    # # return inv_freq, attention_factor
    return factor, low, high, attention_factor


class Qwen3MoeSparseMoeBlock(nn.Module):
    def __init__(
        self,
        layer_id: int,
        config: Qwen3MoeConfig,
        quant_config: Optional[QuantizationConfig] = None,
        prefix: str = "",
    ):
        super().__init__()
        self.tp_size = get_parallel().moe_tp_size
        self.ep_size = get_parallel().moe_ep_size
        self.layer_id = layer_id
        if self.tp_size > config.num_experts:
            raise ValueError(
                f"Tensor parallel size {self.tp_size} is greater than "
                f"the number of experts {config.num_experts}."
            )

        from sglang.srt.layers.quantization.gguf import GGUFConfig

        norm_topk_prob = getattr(config, "norm_topk_prob", True)
        if isinstance(quant_config, GGUFConfig):
            norm_topk_prob = False

        self.topk = TopK(
            top_k=config.num_experts_per_tok,
            renormalize=norm_topk_prob,
            use_grouped_topk=False,
            layer_id=layer_id,
        )

        self.experts = get_moe_impl_class(quant_config)(

View on GitHub (pinned to 0132848349)

Solutions

1. Lower tensor parallel size to <= num_experts
2. Use --ep-size (expert parallel) instead of raising TP beyond expert count
3. Check config.json num_experts and pick a TP/EP factorization where moe_tp_size <= num_experts

Example fix

# before
python -m sglang.launch_server --model Qwen3-30B-A3B --tp 64
# after
python -m sglang.launch_server --model Qwen3-30B-A3B --tp 8 --ep 8

tp, num_experts = get_parallel().moe_tp_size, config.num_experts
assert tp <= num_experts, f"TP {tp} > experts {num_experts}"

When it happens

Trigger: Launching Qwen3-MoE with --tp N (or --moe-tp-size N) where N exceeds config.num_experts in the HF config.json.

Common situations: Running a small-expert MoE checkpoint (e.g. 8 experts A3B) with --tp 16, or mixing --tp with --ep such that effective moe_tp_size > num_experts.

Related errors

• Tensor parallel size {self.tp_size} is greater than the numb
• DeepEP is not installed. Please install DeepEP package from
• The output_size of gate's and up's weight = {intermediate_si
• The input_size of down's weight = {intermediate_size_per_par
• expert-pack v1 supports only single-GPU TP=EP=1

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/922b7e79e20e2782. Report an issue: GitHub.