{"record":{"id":"9ab8813f92215768","repo":"sgl-project/sglang","slug":"shared-sink-lora-pool-shape-changed-after-initiali","errorCode":null,"errorMessage":"Shared-sink LoRA pool shape changed after initialization: gate-up {tuple(self._w1_delta.shape)} -> {expected_gate_up}, down-A {tuple(self._a_cat.shape)} -> {expected_down}","messagePattern":"Shared-sink LoRA pool shape changed after initialization: gate-up (.+?) -> (.+?), down-A (.+?) -> (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/inkling_common/lora.py","lineNumber":104,"sourceCode":"\n    def _allocate_lora_operands(self) -> None:\n        if not self.experts_shared_outer_loras:\n            self._w1_delta = None\n            self._a_cat = None\n            return\n        slots, n, two_f, rank = self.gate_up_lora_b_weights.shape\n        _, _, _, f = self.down_lora_a_weights.shape\n        expected_gate_up = (slots, n * two_f, 2 * rank)\n        expected_down = (slots, rank, n * f)\n        if self._w1_delta is None:\n            self._w1_delta = self.gate_up_lora_b_weights.new_empty(expected_gate_up)\n            self._a_cat = self.down_lora_a_weights.new_empty(expected_down)\n            return\n        if (\n            tuple(self._w1_delta.shape) != expected_gate_up\n            or tuple(self._a_cat.shape) != expected_down\n        ):\n            raise RuntimeError(\n                \"Shared-sink LoRA pool shape changed after initialization: \"\n                f\"gate-up {tuple(self._w1_delta.shape)} -> {expected_gate_up}, \"\n                f\"down-A {tuple(self._a_cat.shape)} -> {expected_down}\"\n            )\n\n    def on_lora_slots_updated(self, slot_ids: set[int] | None) -> None:\n        self._refresh_lora_operands(slot_ids)\n\n    def _refresh_lora_operands(self, slot_ids: set[int] | None = None) -> None:\n        if not self.set_lora or self._w1_delta is None or self._a_cat is None:\n            return\n        b_gate_up = self.gate_up_lora_b_weights\n        a_down = self.down_lora_a_weights\n        slots, n, two_f, rank = b_gate_up.shape\n        f = two_f // 2\n        if slot_ids is None:\n            slot_ids = set(range(slots))\n        elif any(slot < 0 or slot >= slots for slot in slot_ids):","sourceCodeStart":86,"sourceCodeEnd":122,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/inkling_common/lora.py#L86-L122","documentation":"_allocate_lora_operands pre-allocates the shared sink pools (_w1_delta for gate-up, _a_cat for down-A) on first call; on later calls (from set_lora_info as the LoRA pool grows/changes) it verifies the existing pool shapes still match the expected shapes. If the slot count, rank, or expert layout changed between calls, it raises this RuntimeError because reallocation mid-flight would corrupt in-flight batches.","triggerScenarios":"Calling set_lora_info twice with different numbers of LoRA slots or different rank/inner dims — first call allocates pools sized to those buffers, the second call computes different expected shapes (e.g. max_rank changed or more slots added).","commonSituations":"Hot-swapping LoRA adapters of different rank on a live server; loading more adapters than the initially allocated slot count; mixing adapter bundles with inconsistent ranks across updates.","solutions":["Use a consistent slot count and rank across all set_lora_info calls for the life of the sink; allocate for the maximum up front","Restart/recreate the module (or sink) when the LoRA pool shape must change","Normalize all adapters to one rank before registering them"],"exampleFix":"# before\nsink.set_lora_info(a_2d, b_2d, c_2d, d_2d)   # rank 16 -> pools sized r=16\nsink.set_lora_info(a_8, b_8, c_8, d_8)        # rank 8 -> RuntimeError\n# after\n# pre-allocate/serve with one fixed rank and slot count\nassert all(t.shape[-1] == 16 for t in (a, b, c, d))\nsink.set_lora_info(a, b, c, d)","handlingStrategy":"validation","validationCode":"expected_ga = (slots, outer, inner, 2 * max_rank)  # compute once at startup\nif module._w1_delta is not None:\n    assert tuple(module._w1_delta.shape) == expected_gate_up and tuple(module._a_cat.shape) == expected_down\nmodule.set_lora_info(ga, gb, da, db)","typeGuard":"def pool_shapes_stable(module, expected_gate_up, expected_down) -> bool:\n    return (tuple(module._w1_delta.shape) == expected_gate_up\n            and tuple(module._a_cat.shape) == expected_down)","tryCatchPattern":"try: module.set_lora_info(...)\nexcept RuntimeError as e: if 'shape changed' in str(e): recreate the sink with new pools (restart or fresh instance); else: raise","preventionTips":["Fix LoRA rank and slot count for the server's lifetime; allocate for max slots up front","Never hot-swap adapters of different rank on a live sink","Recreate the module when the adapter pool must resize"],"tags":["sglang","lora","pool-allocation","shape-validation","runtime-error","inkling"],"backgroundTag":"state-shape-drift","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}