sgl-project/sglang · error · ValueError

Inkling shared-sink LoRA outer factors must have expert dime

Error message

Inkling shared-sink LoRA outer factors must have expert dimension 1 or {self.n_shared_experts}

What it means

After confirming all four buffers are 4D, set_lora_info checks the outer (expert) factor of gate_up_lora_a_weights.shape[1] and down_lora_b_weights.shape[1]; each must be 1 (shared across experts) or exactly n_shared_experts. Anything else means the expert layout of the supplied buffers disagrees with the module's shared-expert count.

Source

Thrown at python/sglang/srt/models/inkling_common/lora.py:55

        self,
        gate_up_lora_a_weights: torch.Tensor,
        gate_up_lora_b_weights: torch.Tensor,
        down_lora_a_weights: torch.Tensor,
        down_lora_b_weights: torch.Tensor,
    ) -> None:
        tensors = (
            gate_up_lora_a_weights,
            gate_up_lora_b_weights,
            down_lora_a_weights,
            down_lora_b_weights,
        )
        if any(weight.ndim != 4 for weight in tensors):
            raise ValueError("Inkling shared-sink LoRA requires four 4D MoE buffers")
        gate_outer = gate_up_lora_a_weights.shape[1]
        down_outer = down_lora_b_weights.shape[1]
        valid_outer_dims = (1, self.n_shared_experts)
        if gate_outer not in valid_outer_dims or down_outer not in valid_outer_dims:
            raise ValueError(
                "Inkling shared-sink LoRA outer factors must have expert dimension "
                f"1 or {self.n_shared_experts}"
            )
        if gate_outer != down_outer:
            raise ValueError(
                "Inkling shared-sink gate-up A and down B must use the same "
                "expert layout"
            )
        if (
            gate_up_lora_b_weights.shape[1] != self.n_shared_experts
            or down_lora_a_weights.shape[1] != self.n_shared_experts
        ):
            raise ValueError("Inkling shared-sink LoRA expert count does not match")

        max_rank = gate_up_lora_b_weights.shape[-1]
        if (
            gate_up_lora_a_weights.shape[2] != 2 * max_rank
            or down_lora_a_weights.shape[2] != max_rank

View on GitHub (pinned to 0132848349)

Solutions

1. Rebuild/transpose the buffers so dim 1 is 1 or matches the model's n_shared_experts
2. Check the model config's shared-expert count and regenerate the LoRA for that variant
3. Add an assert on shape[1] in your adapter loader before calling set_lora_info

Example fix

# before
# gate_up_lora_a.shape == (slots, num_local_experts, inner, rank)
# after
assert gate_up_lora_a.shape[1] in (1, module.n_shared_experts)
module.set_lora_info(gate_up_lora_a, gate_up_b, down_a, down_b)

valid = (1, module.n_shared_experts)
assert ga.shape[1] in valid and db.shape[1] in valid
module.set_lora_info(ga, gb, da, db)

def outer_dims_valid(module, ga, db) -> bool:
    v = (1, module.n_shared_experts)
    return ga.shape[1] in v and db.shape[1] in v

When it happens

Trigger: Calling set_lora_info where gate_up_lora_a or down_lora_b has shape[1] not in {1, n_shared_experts} — e.g. outer dim equal to the total MoE expert count or an arbitrary rank value.

Common situations: Adapters exported for a different model variant (different number of shared experts); misinterpreting dim 1 as the LoRA rank when packing tensors; model config changed n_shared_experts after adapter conversion.

Related errors

• Inkling shared-sink LoRA expert count does not match
• Inkling shared-sink LoRA requires four 4D MoE buffers
• Inkling shared-sink gate-up A and down B must use the same e
• Inkling shared-sink LoRA rank dimensions do not match
• Shared-sink LoRA pool shape changed after initialization: ga

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c2fb409786ea08bd. Report an issue: GitHub.