{"record":{"id":"4fa0f6f36b95b69c","repo":"sgl-project/sglang","slug":"tensor-parallel-size-self-tp-size-num-experts","errorCode":null,"errorMessage":"Tensor parallel size {self.tp_size} > num_experts {config.num_experts}.","messagePattern":"Tensor parallel size (.+?) > num_experts (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/sdar_moe.py","lineNumber":87,"sourceCode":"    Qwen3MoE-style sparse MoE block:\n      - gate: ReplicatedLinear(hidden, num_experts)\n      - topk routing: TopK\n      - experts: get_moe_impl_class(quant_config)(...)\n    \"\"\"\n\n    def __init__(\n        self,\n        layer_id: int,\n        config: PretrainedConfig,\n        quant_config: Optional[QuantizationConfig] = None,\n        prefix: str = \"\",\n    ):\n        super().__init__()\n        self.layer_id = layer_id\n        self.tp_size = get_parallel().tp_size\n\n        if self.tp_size > config.num_experts:\n            raise ValueError(\n                f\"Tensor parallel size {self.tp_size} > num_experts {config.num_experts}.\"\n            )\n\n        self.topk = TopK(\n            top_k=config.num_experts_per_tok,\n            renormalize=config.norm_topk_prob,\n            use_grouped_topk=False,\n            layer_id=layer_id,\n        )\n\n        self.experts = get_moe_impl_class(quant_config)(\n            num_experts=config.num_experts + get_exec().moe.ep_num_redundant_experts,\n            top_k=config.num_experts_per_tok,\n            layer_id=layer_id,\n            hidden_size=config.hidden_size,\n            intermediate_size=config.moe_intermediate_size,\n            quant_config=quant_config,\n            prefix=add_prefix(\"experts\", prefix),","sourceCodeStart":69,"sourceCodeEnd":105,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/sdar_moe.py#L69-L105","documentation":"SDAR MoE expert weights are partitioned across tensor-parallel ranks, so TP size cannot exceed the number of experts. __init__ validates config.num_experts >= tp_size and fails fast otherwise, because at least one expert per rank is required for the TP split.","triggerScenarios":"Launching an SDAR MoE model with --tp-size N where N > config.num_experts in the model's config.json (e.g. num_experts=8 launched with tp=16).","commonSituations":"Reusing launch scripts written for larger-MoE models (DeepSeek, Qwen-MoE with hundreds of experts) on a small fine-grained MoE checkpoint; misreading num_experts vs num_experts_per_tok in config.json.","solutions":["Reduce --tp-size to <= num_experts from the model's config.json","If more parallelism is needed, combine a smaller TP with --dp-size or expert parallelism (--ep-size) if supported for sdar_moe","Verify you are loading the intended checkpoint whose config.json actually has the expert count you expect"],"exampleFix":"# before\npython -m sglang.launch_server --model sdar-moe --tp-size 16\n# after\npython -m sglang.launch_server --model sdar-moe --tp-size 8","handlingStrategy":"validation","validationCode":"import json\ncfg = json.load(open(\"config.json\"))\ntp = 8\nassert tp <= cfg[\"num_experts\"], f\"tp {tp} > num_experts {cfg['num_experts']}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Read num_experts from config.json before choosing --tp-size","Cache per-model max TP in launch scripts"],"tags":["sdar-moe","tensor-parallel","moe","launch-config"],"backgroundTag":"tp-size-exceeds-experts","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}