{"record":{"id":"d2b1d3fee4ba9313","repo":"sgl-project/sglang","slug":"inkling-shared-sink-gate-up-a-and-down-b-must-use","errorCode":null,"errorMessage":"Inkling shared-sink gate-up A and down B must use the same expert layout","messagePattern":"Inkling shared-sink gate-up A and down B must use the same expert layout","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/inkling_common/lora.py","lineNumber":60,"sourceCode":"    ) -> None:\n        tensors = (\n            gate_up_lora_a_weights,\n            gate_up_lora_b_weights,\n            down_lora_a_weights,\n            down_lora_b_weights,\n        )\n        if any(weight.ndim != 4 for weight in tensors):\n            raise ValueError(\"Inkling shared-sink LoRA requires four 4D MoE buffers\")\n        gate_outer = gate_up_lora_a_weights.shape[1]\n        down_outer = down_lora_b_weights.shape[1]\n        valid_outer_dims = (1, self.n_shared_experts)\n        if gate_outer not in valid_outer_dims or down_outer not in valid_outer_dims:\n            raise ValueError(\n                \"Inkling shared-sink LoRA outer factors must have expert dimension \"\n                f\"1 or {self.n_shared_experts}\"\n            )\n        if gate_outer != down_outer:\n            raise ValueError(\n                \"Inkling shared-sink gate-up A and down B must use the same \"\n                \"expert layout\"\n            )\n        if (\n            gate_up_lora_b_weights.shape[1] != self.n_shared_experts\n            or down_lora_a_weights.shape[1] != self.n_shared_experts\n        ):\n            raise ValueError(\"Inkling shared-sink LoRA expert count does not match\")\n\n        max_rank = gate_up_lora_b_weights.shape[-1]\n        if (\n            gate_up_lora_a_weights.shape[2] != 2 * max_rank\n            or down_lora_a_weights.shape[2] != max_rank\n            or down_lora_b_weights.shape[-1] != max_rank\n        ):\n            raise ValueError(\"Inkling shared-sink LoRA rank dimensions do not match\")\n\n        self.set_lora = True","sourceCodeStart":42,"sourceCodeEnd":78,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/inkling_common/lora.py#L42-L78","documentation":"set_lora_info requires the expert layout choice to be consistent between the gate-up A buffer and the down B buffer: both must use the shared (outer=1) layout or both the per-expert (outer=n_shared_experts) layout. Mixed layouts would make the shared sink's fused math produce wrong results, so it raises immediately.","triggerScenarios":"Calling set_lora_info with gate_up_lora_a_weights.shape[1] == 1 but down_lora_b_weights.shape[1] == n_shared_experts (or vice versa) — buffers packed by different code paths or configs.","commonSituations":"Concatenating adapter shards where one side was expanded to per-expert layout and the other kept shared; partial conversion script applying unsqueeze(1) to only some tensors; version skew between exporter and runtime.","solutions":["Re-pack both buffers with the same outer dim: either both 1 or both n_shared_experts (broadcast/expand the shared one)","Fix the conversion script so it applies the same expert-layout transformation to gate_up A and down B","Assert gate_up_lora_a.shape[1] == down_lora_b.shape[1] before binding"],"exampleFix":"# before\n# gate_up_a: (slots, 1, inner, r); down_b: (slots, E, out, r)\n# after (unify to shared layout)\ndown_b = down_b.expand(slots, 1, out, r) if down_b.shape[1] == 1 else down_b\n# ensure both are (slots, 1, ...) or both (slots, E, ...) before set_lora_info","handlingStrategy":"validation","validationCode":"assert ga.shape[1] == db.shape[1], 'gate-up A and down B expert layouts must match'\nmodule.set_lora_info(ga, gb, da, db)","typeGuard":"def expert_layouts_consistent(ga, db) -> bool: return ga.shape[1] == db.shape[1]","tryCatchPattern":null,"preventionTips":["Use a single packing function for all four buffers","Add shape assertions in the adapter conversion script","Never hand-edit individual buffer dims"],"tags":["sglang","lora","shape-validation","consistency-check","moe","inkling"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}