{"record":{"id":"16bf10112da69073","repo":"sgl-project/sglang","slug":"inkling-shared-sink-lora-expert-count-does-not-mat","errorCode":null,"errorMessage":"Inkling shared-sink LoRA expert count does not match","messagePattern":"Inkling shared-sink LoRA expert count does not match","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/inkling_common/lora.py","lineNumber":68,"sourceCode":"            raise ValueError(\"Inkling shared-sink LoRA requires four 4D MoE buffers\")\n        gate_outer = gate_up_lora_a_weights.shape[1]\n        down_outer = down_lora_b_weights.shape[1]\n        valid_outer_dims = (1, self.n_shared_experts)\n        if gate_outer not in valid_outer_dims or down_outer not in valid_outer_dims:\n            raise ValueError(\n                \"Inkling shared-sink LoRA outer factors must have expert dimension \"\n                f\"1 or {self.n_shared_experts}\"\n            )\n        if gate_outer != down_outer:\n            raise ValueError(\n                \"Inkling shared-sink gate-up A and down B must use the same \"\n                \"expert layout\"\n            )\n        if (\n            gate_up_lora_b_weights.shape[1] != self.n_shared_experts\n            or down_lora_a_weights.shape[1] != self.n_shared_experts\n        ):\n            raise ValueError(\"Inkling shared-sink LoRA expert count does not match\")\n\n        max_rank = gate_up_lora_b_weights.shape[-1]\n        if (\n            gate_up_lora_a_weights.shape[2] != 2 * max_rank\n            or down_lora_a_weights.shape[2] != max_rank\n            or down_lora_b_weights.shape[-1] != max_rank\n        ):\n            raise ValueError(\"Inkling shared-sink LoRA rank dimensions do not match\")\n\n        self.set_lora = True\n        self.gate_up_lora_a_weights = gate_up_lora_a_weights\n        self.gate_up_lora_b_weights = gate_up_lora_b_weights\n        self.down_lora_a_weights = down_lora_a_weights\n        self.down_lora_b_weights = down_lora_b_weights\n        self.experts_shared_outer_loras = gate_outer == 1\n        self._allocate_lora_operands()\n        self._refresh_lora_operands()\n","sourceCodeStart":50,"sourceCodeEnd":86,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/inkling_common/lora.py#L50-L86","documentation":"The inner (expert-side) factors must be per-expert: gate_up_lora_b_weights.shape[1] and down_lora_a_weights.shape[1] must equal n_shared_experts exactly. This complements the outer-dim rule (5375/5376); if either inner factor is 1 or another value, the buffer pair is incompatible with the module's expert count.","triggerScenarios":"Calling set_lora_info where gate_up_lora_b.shape[1] or down_lora_a.shape[1] != n_shared_experts — e.g. both packed as fully-shared (outer=1, inner=1) or built for a different expert count.","commonSituations":"Adapter built for another Inkling variant; conversion script swapped the A/B roles so the expert dim landed on the wrong axis; config mismatch between checkpoint and server args.","solutions":["Re-pack so gate_up B and down A have dim 1 == n_shared_experts (per-expert inner factor)","Regenerate the LoRA adapters against the exact model config being served","Print all four shapes and compare against n_shared_experts before calling set_lora_info"],"exampleFix":"# before\n# gate_up_b.shape == (slots, 1, out, 2r)  # fully shared, inner factor 1\n# after\ngate_up_b = gate_up_b.expand(slots, module.n_shared_experts, out, 2r)\nmodule.set_lora_info(gate_up_a, gate_up_b, down_a, down_b)","handlingStrategy":"validation","validationCode":"assert gb.shape[1] == module.n_shared_experts and da.shape[1] == module.n_shared_experts\nmodule.set_lora_info(ga, gb, da, db)","typeGuard":"def inner_expert_dims_valid(module, gb, da) -> bool:\n    return gb.shape[1] == module.n_shared_experts and da.shape[1] == module.n_shared_experts","tryCatchPattern":null,"preventionTips":["Bind adapters only to the model variant they were exported from","Validate the full 4-tuple of shapes against module constants in one pre-check"],"tags":["sglang","lora","shape-validation","moe","shared-experts","inkling"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}