{"record":{"id":"922b7e79e20e2782","repo":"sgl-project/sglang","slug":"tensor-parallel-size-self-tp-size-is-greater-tha-922b7e","errorCode":null,"errorMessage":"Tensor parallel size {self.tp_size} is greater than the number of experts {config.num_experts}.","messagePattern":"Tensor parallel size (.+?) is greater than the number of experts (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/qwen3_moe.py","lineNumber":241,"sourceCode":"    # # )\n    # # return inv_freq, attention_factor\n    return factor, low, high, attention_factor\n\n\nclass Qwen3MoeSparseMoeBlock(nn.Module):\n    def __init__(\n        self,\n        layer_id: int,\n        config: Qwen3MoeConfig,\n        quant_config: Optional[QuantizationConfig] = None,\n        prefix: str = \"\",\n    ):\n        super().__init__()\n        self.tp_size = get_parallel().moe_tp_size\n        self.ep_size = get_parallel().moe_ep_size\n        self.layer_id = layer_id\n        if self.tp_size > config.num_experts:\n            raise ValueError(\n                f\"Tensor parallel size {self.tp_size} is greater than \"\n                f\"the number of experts {config.num_experts}.\"\n            )\n\n        from sglang.srt.layers.quantization.gguf import GGUFConfig\n\n        norm_topk_prob = getattr(config, \"norm_topk_prob\", True)\n        if isinstance(quant_config, GGUFConfig):\n            norm_topk_prob = False\n\n        self.topk = TopK(\n            top_k=config.num_experts_per_tok,\n            renormalize=norm_topk_prob,\n            use_grouped_topk=False,\n            layer_id=layer_id,\n        )\n\n        self.experts = get_moe_impl_class(quant_config)(","sourceCodeStart":223,"sourceCodeEnd":259,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/qwen3_moe.py#L223-L259","documentation":"Qwen3 MoE requires moe_tp_size <= config.num_experts because expert-parallel/MoE-TP sharding assigns each TP rank at least one expert; with more ranks than experts some rank would own zero experts.","triggerScenarios":"Launching Qwen3-MoE with --tp N (or --moe-tp-size N) where N exceeds config.num_experts in the HF config.json.","commonSituations":"Running a small-expert MoE checkpoint (e.g. 8 experts A3B) with --tp 16, or mixing --tp with --ep such that effective moe_tp_size > num_experts.","solutions":["Lower tensor parallel size to <= num_experts","Use --ep-size (expert parallel) instead of raising TP beyond expert count","Check config.json num_experts and pick a TP/EP factorization where moe_tp_size <= num_experts"],"exampleFix":"# before\npython -m sglang.launch_server --model Qwen3-30B-A3B --tp 64\n# after\npython -m sglang.launch_server --model Qwen3-30B-A3B --tp 8 --ep 8","handlingStrategy":"validation","validationCode":"tp, num_experts = get_parallel().moe_tp_size, config.num_experts\nassert tp <= num_experts, f\"TP {tp} > experts {num_experts}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Factor TP/EP against config.num_experts before launch","Prefer EP for expert-heavy MoE models"],"tags":["moe","tensor-parallel","expert-parallel","qwen3"],"backgroundTag":"parallelism-exceeds-partition-count","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}