sgl-project/sglang · error · ValueError

Inkling shared-sink LoRA expert count does not match

Error message

Inkling shared-sink LoRA expert count does not match

What it means

The inner (expert-side) factors must be per-expert: gate_up_lora_b_weights.shape[1] and down_lora_a_weights.shape[1] must equal n_shared_experts exactly. This complements the outer-dim rule (5375/5376); if either inner factor is 1 or another value, the buffer pair is incompatible with the module's expert count.

Source

Thrown at python/sglang/srt/models/inkling_common/lora.py:68

            raise ValueError("Inkling shared-sink LoRA requires four 4D MoE buffers")
        gate_outer = gate_up_lora_a_weights.shape[1]
        down_outer = down_lora_b_weights.shape[1]
        valid_outer_dims = (1, self.n_shared_experts)
        if gate_outer not in valid_outer_dims or down_outer not in valid_outer_dims:
            raise ValueError(
                "Inkling shared-sink LoRA outer factors must have expert dimension "
                f"1 or {self.n_shared_experts}"
            )
        if gate_outer != down_outer:
            raise ValueError(
                "Inkling shared-sink gate-up A and down B must use the same "
                "expert layout"
            )
        if (
            gate_up_lora_b_weights.shape[1] != self.n_shared_experts
            or down_lora_a_weights.shape[1] != self.n_shared_experts
        ):
            raise ValueError("Inkling shared-sink LoRA expert count does not match")

        max_rank = gate_up_lora_b_weights.shape[-1]
        if (
            gate_up_lora_a_weights.shape[2] != 2 * max_rank
            or down_lora_a_weights.shape[2] != max_rank
            or down_lora_b_weights.shape[-1] != max_rank
        ):
            raise ValueError("Inkling shared-sink LoRA rank dimensions do not match")

        self.set_lora = True
        self.gate_up_lora_a_weights = gate_up_lora_a_weights
        self.gate_up_lora_b_weights = gate_up_lora_b_weights
        self.down_lora_a_weights = down_lora_a_weights
        self.down_lora_b_weights = down_lora_b_weights
        self.experts_shared_outer_loras = gate_outer == 1
        self._allocate_lora_operands()
        self._refresh_lora_operands()

View on GitHub (pinned to 0132848349)

Solutions

1. Re-pack so gate_up B and down A have dim 1 == n_shared_experts (per-expert inner factor)
2. Regenerate the LoRA adapters against the exact model config being served
3. Print all four shapes and compare against n_shared_experts before calling set_lora_info

Example fix

# before
# gate_up_b.shape == (slots, 1, out, 2r)  # fully shared, inner factor 1
# after
gate_up_b = gate_up_b.expand(slots, module.n_shared_experts, out, 2r)
module.set_lora_info(gate_up_a, gate_up_b, down_a, down_b)

assert gb.shape[1] == module.n_shared_experts and da.shape[1] == module.n_shared_experts
module.set_lora_info(ga, gb, da, db)

def inner_expert_dims_valid(module, gb, da) -> bool:
    return gb.shape[1] == module.n_shared_experts and da.shape[1] == module.n_shared_experts

When it happens

Trigger: Calling set_lora_info where gate_up_lora_b.shape[1] or down_lora_a.shape[1] != n_shared_experts — e.g. both packed as fully-shared (outer=1, inner=1) or built for a different expert count.

Common situations: Adapter built for another Inkling variant; conversion script swapped the A/B roles so the expert dim landed on the wrong axis; config mismatch between checkpoint and server args.

Related errors

• Inkling shared-sink LoRA outer factors must have expert dime
• Inkling shared-sink LoRA requires four 4D MoE buffers
• Inkling shared-sink gate-up A and down B must use the same e
• Inkling shared-sink LoRA rank dimensions do not match
• Shared-sink LoRA pool shape changed after initialization: ga

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/16bf10112da69073. Report an issue: GitHub.