{"record":{"id":"8a4410283bbc3587","repo":"sgl-project/sglang","slug":"tensor-parallel-size-self-tp-size-is-greater-tha-8a4410","errorCode":null,"errorMessage":"Tensor parallel size {self.tp_size} is greater than the number of experts {config.num_experts}.","messagePattern":"Tensor parallel size (.+?) is greater than the number of experts (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/lfm2_moe.py","lineNumber":111,"sourceCode":"    - Sigmoid scoring (not softmax) - auxiliary-loss-free style\n    - Expert bias (fp32) for load balancing\n    - Bias affects selection only, not weighting\n    - Uses FusedMoE for efficient batched expert computation\n    \"\"\"\n\n    def __init__(\n        self,\n        config: Lfm2MoeConfig,\n        layer_idx: int,\n        quant_config: Optional[QuantizationConfig] = None,\n        prefix: str = \"\",\n    ):\n        super().__init__()\n        self.tp_size = get_parallel().tp_size\n        self.routed_scaling_factor = config.routed_scaling_factor\n\n        if self.tp_size > config.num_experts:\n            raise ValueError(\n                f\"Tensor parallel size {self.tp_size} is greater than \"\n                f\"the number of experts {config.num_experts}.\"\n            )\n\n        # Gate (router) - outputs logits for each expert\n        self.gate = ReplicatedLinear(\n            config.hidden_size,\n            config.num_experts,\n            bias=False,\n            quant_config=None,\n            prefix=add_prefix(\"gate\", prefix),\n        )\n\n        # Expert bias (fp32) - affects selection but not weighting\n        if config.use_expert_bias:\n            self.expert_bias = nn.Parameter(\n                torch.zeros(config.num_experts, dtype=torch.float32)\n            )","sourceCodeStart":93,"sourceCodeEnd":129,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/lfm2_moe.py#L93-L129","documentation":"LFM2-MoE's expert layer requires tp_size <= num_experts (lfm2_moe.py:111): with tensor parallelism each rank hosts num_experts/tp_size experts, so more ranks than experts leaves some ranks with none and is rejected at init.","triggerScenarios":"Launching LFM2 MoE with --tp-size N where N > config.num_experts (e.g. tp=8 with 6 experts).","commonSituations":"Serving small-expert MoEs on large GPU counts; reusing TP settings from dense models.","solutions":["Reduce --tp-size to <= num_experts","Prefer expert parallel (--ep-size / --enable-ep-moe) to scale across GPUs","Confirm num_experts in the model config before sizing the cluster"],"exampleFix":"# before\n--tp-size 8   (model has 6 experts)\n# after\n--tp-size 4 --enable-ep-moe --dp-size 2","handlingStrategy":"validation","validationCode":"assert args.tp_size <= cfg.num_experts","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Derive TP/EP layout from config (experts, heads) programmatically"],"tags":["lfm2","moe","tensor-parallel","startup-validation"],"backgroundTag":"tp-expert-count-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}