{"record":{"id":"3103d8342cff3c9e","repo":"sgl-project/sglang","slug":"tensor-parallel-size-self-tp-size-is-greater-tha-3103d8","errorCode":null,"errorMessage":"Tensor parallel size {self.tp_size} is greater than the number of experts {config.n_routed_experts}.","messagePattern":"Tensor parallel size (.+?) is greater than the number of experts (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/mimo_v2.py","lineNumber":391,"sourceCode":"\nclass MiMoV2MoE(nn.Module):\n\n    def __init__(\n        self,\n        config: MiMoV2Config,\n        layer_id: int,\n        quant_config: Optional[QuantizationConfig] = None,\n        prefix: str = \"\",\n        is_nextn: bool = False,\n    ):\n        super().__init__()\n        self.tp_size = get_parallel().tp_size\n\n        self.config = config\n        self.layer_id = layer_id\n\n        if self.tp_size > config.n_routed_experts:\n            raise ValueError(\n                f\"Tensor parallel size {self.tp_size} is greater than \"\n                f\"the number of experts {config.n_routed_experts}.\"\n            )\n\n        if config.hidden_act != \"silu\":\n            raise ValueError(\n                f\"Unsupported activation: {config.hidden_act}. \"\n                \"Only silu is supported for now.\"\n            )\n\n        self.gate = MoEGate(\n            config=config,\n            quant_config=quant_config,\n            prefix=add_prefix(\"gate\", prefix),\n            is_nextn=is_nextn,\n        )\n\n        experts_type = get_moe_impl_class(quant_config)","sourceCodeStart":373,"sourceCodeEnd":409,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/mimo_v2.py#L373-L409","documentation":"The MoE block requires tp_size <= number of routed experts because experts are distributed across TP ranks (at least one expert per rank). Exceeding n_routed_experts makes per-rank expert sharding impossible.","triggerScenarios":"Launching MiMo-v2 with --tp greater than config.n_routed_experts, e.g. tp=256 on a model with 128 routed experts.","commonSituations":"Scaling tensor parallelism too aggressively on small-expert MoE models; misreading config where n_routed_experts is smaller than the intended GPU count.","solutions":["Lower --tp to at most config.n_routed_experts","Use expert parallelism (--ep-size) instead of pure TP to scale across more GPUs","Verify n_routed_experts in the config matches the checkpoint"],"exampleFix":"# before\npython -m sglang.launch_server --tp 256 ...\n# after\npython -m sglang.launch_server --tp 8 --ep-size 256 ...","handlingStrategy":"validation","validationCode":"tp = get_parallel().tp_size\nif tp > config.n_routed_experts:\n    raise SystemExit(f'tp={tp} must be <= n_routed_experts={config.n_routed_experts}')","typeGuard":"def tp_supported(tp: int, cfg) -> bool:\n    return tp <= cfg.n_routed_experts","tryCatchPattern":null,"preventionTips":["Cross-check GPU count / --tp against n_routed_experts before launch","Prefer --ep-size for scaling expert count beyond TP limits"],"tags":["moe","tensor-parallel","expert-parallel","launch-config"],"backgroundTag":"parallelism-config-invalid","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}