{"record":{"id":"b2a50ede558e0fa0","repo":"sgl-project/sglang","slug":"inkling-shared-sink-lora-rank-dimensions-do-not-ma","errorCode":null,"errorMessage":"Inkling shared-sink LoRA rank dimensions do not match","messagePattern":"Inkling shared-sink LoRA rank dimensions do not match","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/inkling_common/lora.py","lineNumber":76,"sourceCode":"            )\n        if gate_outer != down_outer:\n            raise ValueError(\n                \"Inkling shared-sink gate-up A and down B must use the same \"\n                \"expert layout\"\n            )\n        if (\n            gate_up_lora_b_weights.shape[1] != self.n_shared_experts\n            or down_lora_a_weights.shape[1] != self.n_shared_experts\n        ):\n            raise ValueError(\"Inkling shared-sink LoRA expert count does not match\")\n\n        max_rank = gate_up_lora_b_weights.shape[-1]\n        if (\n            gate_up_lora_a_weights.shape[2] != 2 * max_rank\n            or down_lora_a_weights.shape[2] != max_rank\n            or down_lora_b_weights.shape[-1] != max_rank\n        ):\n            raise ValueError(\"Inkling shared-sink LoRA rank dimensions do not match\")\n\n        self.set_lora = True\n        self.gate_up_lora_a_weights = gate_up_lora_a_weights\n        self.gate_up_lora_b_weights = gate_up_lora_b_weights\n        self.down_lora_a_weights = down_lora_a_weights\n        self.down_lora_b_weights = down_lora_b_weights\n        self.experts_shared_outer_loras = gate_outer == 1\n        self._allocate_lora_operands()\n        self._refresh_lora_operands()\n\n    def _allocate_lora_operands(self) -> None:\n        if not self.experts_shared_outer_loras:\n            self._w1_delta = None\n            self._a_cat = None\n            return\n        slots, n, two_f, rank = self.gate_up_lora_b_weights.shape\n        _, _, _, f = self.down_lora_a_weights.shape\n        expected_gate_up = (slots, n * two_f, 2 * rank)","sourceCodeStart":58,"sourceCodeEnd":94,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/inkling_common/lora.py#L58-L94","documentation":"set_lora_info derives max_rank from gate_up_lora_b.shape[-1] and requires the rank dims to be consistent: gate_up A's rank axis must be 2*max_rank (fused gate+up), down A's rank axis must be max_rank, and down B's rank axis must be max_rank. Violations mean the A/B rank pairs don't form valid low-rank factorizations.","triggerScenarios":"Calling set_lora_info where gate_up_lora_a.shape[2] != 2 * gate_up_lora_b.shape[-1], or down_lora_a.shape[2] / down_lora_b.shape[-1] differ from max_rank — e.g. gate and up adapters not fused, or adapters of different ranks mixed into one bundle.","commonSituations":"Supplying separate gate and up LoRA matrices instead of the required fused (2r) form; mixing rank-16 and rank-32 adapters in one multi-slot pool; transposition mistakes putting rank on the wrong axis.","solutions":["Fuse gate and up A matrices along the rank axis to size 2*rank before binding","Make all four buffers use one rank r: A pairs (2r, r), B pairs (r, r)","Verify shapes programmatically: a.shape[rank_axis] == (2*)b.shape[-1] for each pair"],"exampleFix":"# before\ngate_up_a = torch.cat([gate_a, up_a], dim=0)   # wrong axis -> rank mismatch\n# after\ngate_up_a = torch.cat([gate_a, up_a], dim=2)   # rank axis -> 2*r as required","handlingStrategy":"validation","validationCode":"r = gb.shape[-1]\nassert ga.shape[2] == 2 * r and da.shape[2] == r and db.shape[-1] == r\nmodule.set_lora_info(ga, gb, da, db)","typeGuard":"def ranks_consistent(ga, gb, da, db) -> bool:\n    r = gb.shape[-1]\n    return ga.shape[2] == 2 * r and da.shape[2] == r and db.shape[-1] == r","tryCatchPattern":null,"preventionTips":["Fuse gate/up A along the rank axis during export","Enforce one rank per adapter bundle","Write a shape contract test for the exporter"],"tags":["sglang","lora","rank-validation","shape-validation","inkling"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}