sgl-project/sglang · error · ValueError

Shared-sink down LoRA-A width must be divisible by {self.n_s

Error message

Shared-sink down LoRA-A width must be divisible by {self.n_shared_experts}, got {flat_intermediate}

What it means

slice_moe_lora_a_weights validates that a 2-D down-proj LoRA-A weight's second dimension (flat intermediate width) is divisible by n_shared_experts before reshaping it into per-shared-expert slices. A non-divisible width cannot be evenly split across shared experts, so the reshape is rejected.

Source

Thrown at python/sglang/srt/models/inkling_common/lora.py:145

            for slot in slot_ids:
                gate_up[slot].zero_()
                gate_up[slot, :, :, 0, :rank].copy_(b_gate_up[slot, :, :f, :])
                gate_up[slot, :, :, 1, rank:].copy_(b_gate_up[slot, :, f:, :])
                a_cat[slot].copy_(a_down[slot].permute(1, 0, 2))

    def slice_moe_lora_a_weights(
        self,
        weights: torch.Tensor | dict[int, torch.Tensor],
        tp_rank: int,
        target_module: str,
    ) -> torch.Tensor | dict[int, torch.Tensor]:
        if isinstance(weights, torch.Tensor) and weights.dim() == 2:
            if target_module == "gate_up_proj_moe":
                weights = weights.unsqueeze(0)
            else:
                rank, flat_intermediate = weights.shape
                if flat_intermediate % self.n_shared_experts != 0:
                    raise ValueError(
                        "Shared-sink down LoRA-A width must be divisible by "
                        f"{self.n_shared_experts}, got {flat_intermediate}"
                    )
                weights = (
                    weights.view(
                        rank,
                        self.n_shared_experts,
                        flat_intermediate // self.n_shared_experts,
                    )
                    .transpose(0, 1)
                    .contiguous()
                )
        if self.moe_tp_size <= 1 or target_module != "down_proj_moe":
            return weights
        if isinstance(weights, dict):
            return {
                expert_id: self._slice_down_lora_a(weight, tp_rank)
                for expert_id, weight in weights.items()

View on GitHub (pinned to 0132848349)

Solutions

1. Verify the adapter's LoRA-A width equals intermediate_size (which is divisible by n_shared_experts) of the target model
2. Retrain or re-export the LoRA adapter against the same model config (n_shared_experts and intermediate size)
3. Route gate_up modules through gate_up_proj_moe so they are unsqueezed instead of validated against n_shared_experts

Example fix

# before
lora_a = torch.randn(rank, 3072)  # model has 2 shared experts, 3072 % 2 -> ok only if divisible; e.g. 3073 fails
# after
assert lora_a.shape[1] % model.n_shared_experts == 0
lora_a = torch.randn(rank, model.intermediate_size)

if lora_a.dim() == 2 and lora_a.shape[1] % model.n_shared_experts != 0:
    raise ValueError('bad LoRA-A width before load')

def is_valid_lora_a(w, n_shared):
    return w.dim() == 2 and w.shape[1] % n_shared == 0

When it happens

Trigger: Loading a LoRA adapter whose down_proj LoRA-A matrix has flat_intermediate % n_shared_experts != 0 while target_module is not gate_up_proj_moe.

Common situations: Adapter checkpoint trained for a different shared-expert count or intermediate size than the serving model; exporting LoRA with a mismatched rank/width layout.

Related errors

• Shared-sink gate/up LoRA-B height must be divisible by {self
• Mixed shared-outer LoRA formats detected across loaded adapt
• {gate_name} has shape {tuple(g.shape)}, expected ({H * D}, {
• The output_size of gate's and up's weight = {intermediate_si
• The input_size of down's weight = {intermediate_size_per_par

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2f83602954fa267e. Report an issue: GitHub.