{"record":{"id":"13703f314522c1bc","repo":"sgl-project/sglang","slug":"shared-sink-gate-up-lora-b-height-must-be-divisibl","errorCode":null,"errorMessage":"Shared-sink gate/up LoRA-B height must be divisible by {self.n_shared_experts}, got {flat_intermediate}","messagePattern":"Shared-sink gate/up LoRA-B height must be divisible by (.+?), got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/inkling_common/lora.py","lineNumber":184,"sourceCode":"\n    def _slice_down_lora_a(self, weights: torch.Tensor, tp_rank: int) -> torch.Tensor:\n        start = tp_rank * self.intermediate_size_per_partition\n        end = start + self.intermediate_size_per_partition\n        return weights[..., start:end].contiguous()\n\n    def slice_moe_lora_b_weights(\n        self,\n        weights: torch.Tensor | dict[int, torch.Tensor],\n        tp_rank: int,\n        target_module: str,\n    ) -> torch.Tensor | dict[int, torch.Tensor]:\n        if isinstance(weights, torch.Tensor) and weights.dim() == 2:\n            if target_module == \"down_proj_moe\":\n                weights = weights.unsqueeze(0)\n            else:\n                flat_intermediate, rank = weights.shape\n                if flat_intermediate % self.n_shared_experts != 0:\n                    raise ValueError(\n                        \"Shared-sink gate/up LoRA-B height must be divisible by \"\n                        f\"{self.n_shared_experts}, got {flat_intermediate}\"\n                    )\n                weights = weights.view(\n                    self.n_shared_experts,\n                    flat_intermediate // self.n_shared_experts,\n                    rank,\n                )\n        if self.moe_tp_size <= 1 or target_module != \"gate_up_proj_moe\":\n            return weights\n        if isinstance(weights, dict):\n            return {\n                expert_id: self._slice_gate_up_lora_b(weight, tp_rank)\n                for expert_id, weight in weights.items()\n            }\n        if weights.dim() == 3:\n            return torch.stack(\n                [","sourceCodeStart":166,"sourceCodeEnd":202,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/inkling_common/lora.py#L166-L202","documentation":"slice_moe_lora_b_weights requires a 2-D gate/up LoRA-B weight's first dimension (flat intermediate height) to be divisible by n_shared_experts so it can be viewed as (n_shared_experts, flat_intermediate // n_shared_experts, rank). Non-divisible heights cannot be partitioned per shared expert.","triggerScenarios":"Loading a gate/up LoRA-B tensor with flat_intermediate % n_shared_experts != 0 while target_module is not down_proj_moe.","commonSituations":"Adapter exported with a different intermediate size or expert grouping than the served Inkling model; wrong tensor passed to the B-slicing path.","solutions":["Confirm the LoRA-B height matches the model's flattened shared-expert intermediate width (divisible by n_shared_experts)","Re-export the adapter from the same base model config","For down-proj tensors, pass target_module='down_proj_moe' so they are unsqueezed rather than divisibility-checked"],"exampleFix":"# before\nlora_b = torch.randn(3073, rank)  # not divisible by n_shared_experts=2\n# after\nlora_b = torch.randn(2 * (model.intermediate_size // 2), rank)","handlingStrategy":"validation","validationCode":"if lora_b.dim() == 2 and lora_b.shape[0] % model.n_shared_experts != 0:\n    raise ValueError('bad LoRA-B height before load')","typeGuard":"def is_valid_lora_b(w, n_shared):\n    return w.dim() == 2 and w.shape[0] % n_shared == 0","tryCatchPattern":null,"preventionTips":["Shape-check adapters in a preflight step","Keep n_shared_experts consistent between training and serving"],"tags":["lora","shape-mismatch","weight-loading","moe"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}