sgl-project/sglang · error · RuntimeError

Shared-sink LoRA pool shape changed after initialization: ga

Error message

Shared-sink LoRA pool shape changed after initialization: gate-up {tuple(self._w1_delta.shape)} -> {expected_gate_up}, down-A {tuple(self._a_cat.shape)} -> {expected_down}

What it means

_allocate_lora_operands pre-allocates the shared sink pools (_w1_delta for gate-up, _a_cat for down-A) on first call; on later calls (from set_lora_info as the LoRA pool grows/changes) it verifies the existing pool shapes still match the expected shapes. If the slot count, rank, or expert layout changed between calls, it raises this RuntimeError because reallocation mid-flight would corrupt in-flight batches.

Source

Thrown at python/sglang/srt/models/inkling_common/lora.py:104

    def _allocate_lora_operands(self) -> None:
        if not self.experts_shared_outer_loras:
            self._w1_delta = None
            self._a_cat = None
            return
        slots, n, two_f, rank = self.gate_up_lora_b_weights.shape
        _, _, _, f = self.down_lora_a_weights.shape
        expected_gate_up = (slots, n * two_f, 2 * rank)
        expected_down = (slots, rank, n * f)
        if self._w1_delta is None:
            self._w1_delta = self.gate_up_lora_b_weights.new_empty(expected_gate_up)
            self._a_cat = self.down_lora_a_weights.new_empty(expected_down)
            return
        if (
            tuple(self._w1_delta.shape) != expected_gate_up
            or tuple(self._a_cat.shape) != expected_down
        ):
            raise RuntimeError(
                "Shared-sink LoRA pool shape changed after initialization: "
                f"gate-up {tuple(self._w1_delta.shape)} -> {expected_gate_up}, "
                f"down-A {tuple(self._a_cat.shape)} -> {expected_down}"
            )

    def on_lora_slots_updated(self, slot_ids: set[int] | None) -> None:
        self._refresh_lora_operands(slot_ids)

    def _refresh_lora_operands(self, slot_ids: set[int] | None = None) -> None:
        if not self.set_lora or self._w1_delta is None or self._a_cat is None:
            return
        b_gate_up = self.gate_up_lora_b_weights
        a_down = self.down_lora_a_weights
        slots, n, two_f, rank = b_gate_up.shape
        f = two_f // 2
        if slot_ids is None:
            slot_ids = set(range(slots))
        elif any(slot < 0 or slot >= slots for slot in slot_ids):

View on GitHub (pinned to 0132848349)

Solutions

1. Use a consistent slot count and rank across all set_lora_info calls for the life of the sink; allocate for the maximum up front
2. Restart/recreate the module (or sink) when the LoRA pool shape must change
3. Normalize all adapters to one rank before registering them

Example fix

# before
sink.set_lora_info(a_2d, b_2d, c_2d, d_2d)   # rank 16 -> pools sized r=16
sink.set_lora_info(a_8, b_8, c_8, d_8)        # rank 8 -> RuntimeError
# after
# pre-allocate/serve with one fixed rank and slot count
assert all(t.shape[-1] == 16 for t in (a, b, c, d))
sink.set_lora_info(a, b, c, d)

expected_ga = (slots, outer, inner, 2 * max_rank)  # compute once at startup
if module._w1_delta is not None:
    assert tuple(module._w1_delta.shape) == expected_gate_up and tuple(module._a_cat.shape) == expected_down
module.set_lora_info(ga, gb, da, db)

def pool_shapes_stable(module, expected_gate_up, expected_down) -> bool:
    return (tuple(module._w1_delta.shape) == expected_gate_up
            and tuple(module._a_cat.shape) == expected_down)

try: module.set_lora_info(...)
except RuntimeError as e: if 'shape changed' in str(e): recreate the sink with new pools (restart or fresh instance); else: raise

When it happens

Trigger: Calling set_lora_info twice with different numbers of LoRA slots or different rank/inner dims — first call allocates pools sized to those buffers, the second call computes different expected shapes (e.g. max_rank changed or more slots added).

Common situations: Hot-swapping LoRA adapters of different rank on a live server; loading more adapters than the initially allocated slot count; mixing adapter bundles with inconsistent ranks across updates.

Related errors

• Inkling shared-sink LoRA requires four 4D MoE buffers
• Inkling shared-sink LoRA outer factors must have expert dime
• Inkling shared-sink gate-up A and down B must use the same e
• Inkling shared-sink LoRA expert count does not match
• Inkling shared-sink LoRA rank dimensions do not match

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9ab8813f92215768. Report an issue: GitHub.