sgl-project/sglang · error · ValueError

Inkling shared-sink LoRA requires four 4D MoE buffers

Error message

Inkling shared-sink LoRA requires four 4D MoE buffers

What it means

set_lora_info validates the four MoE LoRA weight buffers and requires each to be a 4D tensor of shape (slots, outer, inner, rank)-style layout. If any of gate_up_lora_a/b or down_lora_a/b is not 4D, it raises this ValueError, because the shared-expert sink indexes them as batched 4D pools.

Source

Thrown at python/sglang/srt/models/inkling_common/lora.py:50

        self.register_buffer("_a_cat", None, persistent=False)
        self._lora_routing_cache = {}
        lora_backend.is_moe_lora = True

    def set_lora_info(
        self,
        gate_up_lora_a_weights: torch.Tensor,
        gate_up_lora_b_weights: torch.Tensor,
        down_lora_a_weights: torch.Tensor,
        down_lora_b_weights: torch.Tensor,
    ) -> None:
        tensors = (
            gate_up_lora_a_weights,
            gate_up_lora_b_weights,
            down_lora_a_weights,
            down_lora_b_weights,
        )
        if any(weight.ndim != 4 for weight in tensors):
            raise ValueError("Inkling shared-sink LoRA requires four 4D MoE buffers")
        gate_outer = gate_up_lora_a_weights.shape[1]
        down_outer = down_lora_b_weights.shape[1]
        valid_outer_dims = (1, self.n_shared_experts)
        if gate_outer not in valid_outer_dims or down_outer not in valid_outer_dims:
            raise ValueError(
                "Inkling shared-sink LoRA outer factors must have expert dimension "
                f"1 or {self.n_shared_experts}"
            )
        if gate_outer != down_outer:
            raise ValueError(
                "Inkling shared-sink gate-up A and down B must use the same "
                "expert layout"
            )
        if (
            gate_up_lora_b_weights.shape[1] != self.n_shared_experts
            or down_lora_a_weights.shape[1] != self.n_shared_experts
        ):
            raise ValueError("Inkling shared-sink LoRA expert count does not match")

View on GitHub (pinned to 0132848349)

Solutions

1. Reshape/expand the adapters to 4D MoE layout: (num_slots, outer, inner, rank) before calling set_lora_info
2. Re-export the LoRA with the Inkling multi-slot dense converter so buffers carry the slot and expert dims
3. Verify each of the four tensors with weight.ndim == 4 before binding

Example fix

# before
set_lora_info(gate_up_a, gate_up_b, down_a, down_b)  # 2D dense LoRA tensors
# after
gate_up_a4 = gate_up_a.unsqueeze(0).unsqueeze(0)  # add slot & expert dims -> 4D
set_lora_info(gate_up_a4, gate_up_b4, down_a4, down_b4)

tensors = (ga, gb, da, db)
assert all(getattr(t, 'ndim', 0) == 4 for t in tensors), 'need 4D MoE LoRA buffers'
module.set_lora_info(ga, gb, da, db)

def is_4d_moe_lora(*ts) -> bool:
    return all(hasattr(t, 'ndim') and t.ndim == 4 for t in ts)

except ValueError as e: raise TypeError('adapter is not multi-slot MoE LoRA; re-export as 4D buffers') from e

When it happens

Trigger: Calling set_lora_info with LoRA weights that are 2D (standard dense LoRA A/B of shape (r,in)/(out,r)) or 3D (single-batch per-request LoRA) instead of 4D multi-slot MoE buffers.

Common situations: Loading a regular single-adapter LoRA checkpoint into the Inkling shared-sink path; adapter conversion script dropped the slot/expert dimension; mixing standard LoRA request flow with the batched dense MLP LoRA pool.

Related errors

• Inkling shared-sink LoRA outer factors must have expert dime
• Inkling shared-sink gate-up A and down B must use the same e
• Inkling shared-sink LoRA expert count does not match
• Inkling shared-sink LoRA rank dimensions do not match
• Shared-sink LoRA pool shape changed after initialization: ga

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1209b8229d0ec4ae. Report an issue: GitHub.