{"record":{"id":"92c1141b2560b478","repo":"sgl-project/sglang","slug":"tensor-parallel-size-self-tp-size-is-greater-tha","errorCode":null,"errorMessage":"Tensor parallel size {self.tp_size} is greater than the number of experts {self.n_routed_experts}.","messagePattern":"Tensor parallel size (.+?) is greater than the number of experts (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/models/afmoe.py","lineNumber":170,"sourceCode":"        return topk_weights.to(torch.float32), topk_ids.to(torch.int32)\n\n    def __init__(\n        self,\n        config: PretrainedConfig,\n        quant_config: Optional[QuantizationConfig] = None,\n        prefix: str = \"\",\n    ):\n        super().__init__()\n        self.config = config\n        self.rank = get_parallel().tp_rank\n        self.tp_size = get_parallel().tp_size\n\n        self.n_routed_experts = getattr(config, \"num_experts\", None)\n        if self.n_routed_experts is None:\n            raise ValueError(\"AfmoeConfig must define `num_experts`.\")\n        self.top_k = config.num_experts_per_tok\n        if self.tp_size > self.n_routed_experts:\n            raise ValueError(\n                f\"Tensor parallel size {self.tp_size} is greater than \"\n                f\"the number of experts {self.n_routed_experts}.\"\n            )\n\n        self.score_func = getattr(config, \"score_func\", \"softmax\")\n        self.route_norm = getattr(config, \"route_norm\", True)\n        self.route_scale = float(getattr(config, \"route_scale\", 1.0))\n        self.n_group = getattr(config, \"n_group\", 1)\n        self.topk_group = getattr(config, \"topk_group\", 1)\n        self.use_grouped_topk = self.n_group is not None and self.n_group > 1\n        self.num_shared_experts = getattr(config, \"num_shared_experts\", 0)\n\n        self.gate = ReplicatedLinear(\n            config.hidden_size,\n            self.n_routed_experts,\n            bias=False,\n            quant_config=None,\n            prefix=add_prefix(\"gate\", prefix),","sourceCodeStart":152,"sourceCodeEnd":188,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/afmoe.py#L152-L188","documentation":"AFMoE shards experts across tensor-parallel ranks, so you cannot have more TP ranks than experts. The check fails when tp_size > num_experts because some rank would own zero experts.","triggerScenarios":"Launching with --tp 8 on an AFmoe model whose config has fewer than 8 num_experts; combining large TP with a small expert count.","commonSituations":"Reusing a high-TP launch command from a dense model on a small MoE model; misreading num_experts_per_tok as num_experts.","solutions":["Reduce --tensor-parallel-size to <= num_experts (e.g. tp=2 for 4 experts)","Increase the model's num_experts by using a different checkpoint if more parallelism is required"],"exampleFix":"# before\npython -m sglang.launch_server --model afmoe-x --tp 8\n# after\npython -m sglang.launch_server --model afmoe-x --tp 2","handlingStrategy":"validation","validationCode":"assert tp_size <= config.num_experts, f\"tp {tp_size} > experts {config.num_experts}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check expert count vs --tp before launch","Script a preflight that reads config.json and validates parallelism args"],"tags":["moe","tensor-parallel","afmoe","startup"],"backgroundTag":"parallelism-config-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}