sgl-project/sglang · error · ValueError

Tensor parallel size {self.tp_size} > num_experts {config.nu

Error message

Tensor parallel size {self.tp_size} > num_experts {config.num_experts}.

What it means

SDAR MoE expert weights are partitioned across tensor-parallel ranks, so TP size cannot exceed the number of experts. __init__ validates config.num_experts >= tp_size and fails fast otherwise, because at least one expert per rank is required for the TP split.

Source

Thrown at python/sglang/srt/models/sdar_moe.py:87

    Qwen3MoE-style sparse MoE block:
      - gate: ReplicatedLinear(hidden, num_experts)
      - topk routing: TopK
      - experts: get_moe_impl_class(quant_config)(...)
    """

    def __init__(
        self,
        layer_id: int,
        config: PretrainedConfig,
        quant_config: Optional[QuantizationConfig] = None,
        prefix: str = "",
    ):
        super().__init__()
        self.layer_id = layer_id
        self.tp_size = get_parallel().tp_size

        if self.tp_size > config.num_experts:
            raise ValueError(
                f"Tensor parallel size {self.tp_size} > num_experts {config.num_experts}."
            )

        self.topk = TopK(
            top_k=config.num_experts_per_tok,
            renormalize=config.norm_topk_prob,
            use_grouped_topk=False,
            layer_id=layer_id,
        )

        self.experts = get_moe_impl_class(quant_config)(
            num_experts=config.num_experts + get_exec().moe.ep_num_redundant_experts,
            top_k=config.num_experts_per_tok,
            layer_id=layer_id,
            hidden_size=config.hidden_size,
            intermediate_size=config.moe_intermediate_size,
            quant_config=quant_config,
            prefix=add_prefix("experts", prefix),

View on GitHub (pinned to 0132848349)

Solutions

1. Reduce --tp-size to <= num_experts from the model's config.json
2. If more parallelism is needed, combine a smaller TP with --dp-size or expert parallelism (--ep-size) if supported for sdar_moe
3. Verify you are loading the intended checkpoint whose config.json actually has the expert count you expect

Example fix

# before
python -m sglang.launch_server --model sdar-moe --tp-size 16
# after
python -m sglang.launch_server --model sdar-moe --tp-size 8

import json
cfg = json.load(open("config.json"))
tp = 8
assert tp <= cfg["num_experts"], f"tp {tp} > num_experts {cfg['num_experts']}"

When it happens

Trigger: Launching an SDAR MoE model with --tp-size N where N > config.num_experts in the model's config.json (e.g. num_experts=8 launched with tp=16).

Common situations: Reusing launch scripts written for larger-MoE models (DeepSeek, Qwen-MoE with hundreds of experts) on a small fine-grained MoE checkpoint; misreading num_experts vs num_experts_per_tok in config.json.

Related errors

• Tensor parallel size {self.tp_size} is greater than the numb
• The output_size of gate's and up's weight = {intermediate_si
• The input_size of down's weight = {intermediate_size_per_par
• expert-pack v1 supports only single-GPU TP=EP=1
• The output_size of gate's and up's weight = {intermediate_si

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4fa0f6f36b95b69c. Report an issue: GitHub.