sgl-project/sglang · error · ValueError

Inkling shared-sink gate-up A and down B must use the same e

Error message

Inkling shared-sink gate-up A and down B must use the same expert layout

What it means

set_lora_info requires the expert layout choice to be consistent between the gate-up A buffer and the down B buffer: both must use the shared (outer=1) layout or both the per-expert (outer=n_shared_experts) layout. Mixed layouts would make the shared sink's fused math produce wrong results, so it raises immediately.

Source

Thrown at python/sglang/srt/models/inkling_common/lora.py:60

    ) -> None:
        tensors = (
            gate_up_lora_a_weights,
            gate_up_lora_b_weights,
            down_lora_a_weights,
            down_lora_b_weights,
        )
        if any(weight.ndim != 4 for weight in tensors):
            raise ValueError("Inkling shared-sink LoRA requires four 4D MoE buffers")
        gate_outer = gate_up_lora_a_weights.shape[1]
        down_outer = down_lora_b_weights.shape[1]
        valid_outer_dims = (1, self.n_shared_experts)
        if gate_outer not in valid_outer_dims or down_outer not in valid_outer_dims:
            raise ValueError(
                "Inkling shared-sink LoRA outer factors must have expert dimension "
                f"1 or {self.n_shared_experts}"
            )
        if gate_outer != down_outer:
            raise ValueError(
                "Inkling shared-sink gate-up A and down B must use the same "
                "expert layout"
            )
        if (
            gate_up_lora_b_weights.shape[1] != self.n_shared_experts
            or down_lora_a_weights.shape[1] != self.n_shared_experts
        ):
            raise ValueError("Inkling shared-sink LoRA expert count does not match")

        max_rank = gate_up_lora_b_weights.shape[-1]
        if (
            gate_up_lora_a_weights.shape[2] != 2 * max_rank
            or down_lora_a_weights.shape[2] != max_rank
            or down_lora_b_weights.shape[-1] != max_rank
        ):
            raise ValueError("Inkling shared-sink LoRA rank dimensions do not match")

        self.set_lora = True

View on GitHub (pinned to 0132848349)

Solutions

1. Re-pack both buffers with the same outer dim: either both 1 or both n_shared_experts (broadcast/expand the shared one)
2. Fix the conversion script so it applies the same expert-layout transformation to gate_up A and down B
3. Assert gate_up_lora_a.shape[1] == down_lora_b.shape[1] before binding

Example fix

# before
# gate_up_a: (slots, 1, inner, r); down_b: (slots, E, out, r)
# after (unify to shared layout)
down_b = down_b.expand(slots, 1, out, r) if down_b.shape[1] == 1 else down_b
# ensure both are (slots, 1, ...) or both (slots, E, ...) before set_lora_info

assert ga.shape[1] == db.shape[1], 'gate-up A and down B expert layouts must match'
module.set_lora_info(ga, gb, da, db)

def expert_layouts_consistent(ga, db) -> bool: return ga.shape[1] == db.shape[1]

When it happens

Trigger: Calling set_lora_info with gate_up_lora_a_weights.shape[1] == 1 but down_lora_b_weights.shape[1] == n_shared_experts (or vice versa) — buffers packed by different code paths or configs.

Common situations: Concatenating adapter shards where one side was expanded to per-expert layout and the other kept shared; partial conversion script applying unsqueeze(1) to only some tensors; version skew between exporter and runtime.

Related errors

• Inkling shared-sink LoRA requires four 4D MoE buffers
• Inkling shared-sink LoRA outer factors must have expert dime
• Inkling shared-sink LoRA expert count does not match
• Inkling shared-sink LoRA rank dimensions do not match
• Shared-sink LoRA pool shape changed after initialization: ga

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d2b1d3fee4ba9313. Report an issue: GitHub.