{"record":{"id":"4a6cf6c8ef57085e","repo":"sgl-project/sglang","slug":"tensor-parallel-size-self-tp-size-is-greater-tha-4a6cf6","errorCode":null,"errorMessage":"Tensor parallel size {self.tp_size} is greater than the number of experts {config.moe_num_experts}.","messagePattern":"Tensor parallel size (.+?) is greater than the number of experts (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/models/step3p5.py","lineNumber":131,"sourceCode":"        layer_id: int,\n        quant_config: Optional[QuantizationConfig] = None,\n        prefix: str = \"\",\n    ):\n        super().__init__()\n        self.tp_size = get_parallel().tp_size\n        self.layer_id = layer_id\n\n        self.need_fp32_gate = config.need_fp32_gate\n        self.routed_scaling_factor = config.moe_router_scaling_factor\n        self.use_moe_router_bias = config.use_moe_router_bias\n        if self.use_moe_router_bias:\n            self.router_bias = nn.Parameter(\n                torch.zeros(config.moe_num_experts, dtype=torch.float32),\n                requires_grad=False,\n            )\n\n        if self.tp_size > config.moe_num_experts:\n            raise ValueError(\n                f\"Tensor parallel size {self.tp_size} is greater than \"\n                f\"the number of experts {config.moe_num_experts}.\"\n            )\n\n        self.limit = config.swiglu_limits[layer_id]\n        self.limit = self.limit if self.limit > 0 else None\n\n        self.topk = TopK(\n            top_k=config.moe_top_k,\n            renormalize=True,\n            use_grouped_topk=False,\n            scoring_func=\"sigmoid\",\n            correction_bias=self.router_bias,\n            apply_routed_scaling_factor_on_output=False,\n            layer_id=layer_id,\n        )\n\n        self.experts = get_moe_impl_class(quant_config)(","sourceCodeStart":113,"sourceCodeEnd":149,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/step3p5.py#L113-L149","documentation":"The MoE gate in Step3p5 requires at least one expert per tensor-parallel rank. If TP size exceeds moe_num_experts, expert weights cannot be sharded and the model raises during __init__.","triggerScenarios":"Launching with --tp-size N where N > config.moe_num_experts (e.g. tp=8 on a 4-expert MoE layer).","commonSituations":"Reusing a large-TP launch command on a small dense/MoE variant; misreading expert count from config; A100/H100 multi-node scripts applied to a tiny test checkpoint.","solutions":["Reduce --tensor-parallel-size to <= config.moe_num_experts","Check config.moe_num_experts in the model's config.json to confirm the true expert count","Use expert parallelism (--ep-size) instead if you need more GPUs than experts"],"exampleFix":"# before\npython -m sglang.launch_server --model step3p5 --tp 8\n# after (assuming 4 experts)\npython -m sglang.launch_server --model step3p5 --tp 4","handlingStrategy":"validation","validationCode":"import json\ncfg = json.load(open('config.json'))\ntp = 4\nassert tp <= cfg['moe_num_experts'], f\"tp {tp} > experts {cfg['moe_num_experts']}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate tp_size against config before launch","Prefer --ep-size when GPU count exceeds expert count","Add startup assertion in launch scripts"],"tags":["tensor-parallel","moe","config-validation"],"backgroundTag":"tp-size-exceeds-experts","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}