{"record":{"id":"1209b8229d0ec4ae","repo":"sgl-project/sglang","slug":"inkling-shared-sink-lora-requires-four-4d-moe-buff","errorCode":null,"errorMessage":"Inkling shared-sink LoRA requires four 4D MoE buffers","messagePattern":"Inkling shared-sink LoRA requires four 4D MoE buffers","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/inkling_common/lora.py","lineNumber":50,"sourceCode":"        self.register_buffer(\"_a_cat\", None, persistent=False)\n        self._lora_routing_cache = {}\n        lora_backend.is_moe_lora = True\n\n    def set_lora_info(\n        self,\n        gate_up_lora_a_weights: torch.Tensor,\n        gate_up_lora_b_weights: torch.Tensor,\n        down_lora_a_weights: torch.Tensor,\n        down_lora_b_weights: torch.Tensor,\n    ) -> None:\n        tensors = (\n            gate_up_lora_a_weights,\n            gate_up_lora_b_weights,\n            down_lora_a_weights,\n            down_lora_b_weights,\n        )\n        if any(weight.ndim != 4 for weight in tensors):\n            raise ValueError(\"Inkling shared-sink LoRA requires four 4D MoE buffers\")\n        gate_outer = gate_up_lora_a_weights.shape[1]\n        down_outer = down_lora_b_weights.shape[1]\n        valid_outer_dims = (1, self.n_shared_experts)\n        if gate_outer not in valid_outer_dims or down_outer not in valid_outer_dims:\n            raise ValueError(\n                \"Inkling shared-sink LoRA outer factors must have expert dimension \"\n                f\"1 or {self.n_shared_experts}\"\n            )\n        if gate_outer != down_outer:\n            raise ValueError(\n                \"Inkling shared-sink gate-up A and down B must use the same \"\n                \"expert layout\"\n            )\n        if (\n            gate_up_lora_b_weights.shape[1] != self.n_shared_experts\n            or down_lora_a_weights.shape[1] != self.n_shared_experts\n        ):\n            raise ValueError(\"Inkling shared-sink LoRA expert count does not match\")","sourceCodeStart":32,"sourceCodeEnd":68,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/inkling_common/lora.py#L32-L68","documentation":"set_lora_info validates the four MoE LoRA weight buffers and requires each to be a 4D tensor of shape (slots, outer, inner, rank)-style layout. If any of gate_up_lora_a/b or down_lora_a/b is not 4D, it raises this ValueError, because the shared-expert sink indexes them as batched 4D pools.","triggerScenarios":"Calling set_lora_info with LoRA weights that are 2D (standard dense LoRA A/B of shape (r,in)/(out,r)) or 3D (single-batch per-request LoRA) instead of 4D multi-slot MoE buffers.","commonSituations":"Loading a regular single-adapter LoRA checkpoint into the Inkling shared-sink path; adapter conversion script dropped the slot/expert dimension; mixing standard LoRA request flow with the batched dense MLP LoRA pool.","solutions":["Reshape/expand the adapters to 4D MoE layout: (num_slots, outer, inner, rank) before calling set_lora_info","Re-export the LoRA with the Inkling multi-slot dense converter so buffers carry the slot and expert dims","Verify each of the four tensors with weight.ndim == 4 before binding"],"exampleFix":"# before\nset_lora_info(gate_up_a, gate_up_b, down_a, down_b)  # 2D dense LoRA tensors\n# after\ngate_up_a4 = gate_up_a.unsqueeze(0).unsqueeze(0)  # add slot & expert dims -> 4D\nset_lora_info(gate_up_a4, gate_up_b4, down_a4, down_b4)","handlingStrategy":"type-guard","validationCode":"tensors = (ga, gb, da, db)\nassert all(getattr(t, 'ndim', 0) == 4 for t in tensors), 'need 4D MoE LoRA buffers'\nmodule.set_lora_info(ga, gb, da, db)","typeGuard":"def is_4d_moe_lora(*ts) -> bool:\n    return all(hasattr(t, 'ndim') and t.ndim == 4 for t in ts)","tryCatchPattern":"except ValueError as e: raise TypeError('adapter is not multi-slot MoE LoRA; re-export as 4D buffers') from e","preventionTips":["Always convert adapters with the Inkling multi-slot exporter","Standardize on (slots, outer, inner, rank) layout in tooling","Unit-test adapter loader shapes"],"tags":["sglang","lora","shape-validation","moe","inkling"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}