sgl-project/sglang · error · ValueError

Shared-sink gate/up LoRA-B height must be divisible by {self

Error message

Shared-sink gate/up LoRA-B height must be divisible by {self.n_shared_experts}, got {flat_intermediate}

What it means

slice_moe_lora_b_weights requires a 2-D gate/up LoRA-B weight's first dimension (flat intermediate height) to be divisible by n_shared_experts so it can be viewed as (n_shared_experts, flat_intermediate // n_shared_experts, rank). Non-divisible heights cannot be partitioned per shared expert.

Source

Thrown at python/sglang/srt/models/inkling_common/lora.py:184

    def _slice_down_lora_a(self, weights: torch.Tensor, tp_rank: int) -> torch.Tensor:
        start = tp_rank * self.intermediate_size_per_partition
        end = start + self.intermediate_size_per_partition
        return weights[..., start:end].contiguous()

    def slice_moe_lora_b_weights(
        self,
        weights: torch.Tensor | dict[int, torch.Tensor],
        tp_rank: int,
        target_module: str,
    ) -> torch.Tensor | dict[int, torch.Tensor]:
        if isinstance(weights, torch.Tensor) and weights.dim() == 2:
            if target_module == "down_proj_moe":
                weights = weights.unsqueeze(0)
            else:
                flat_intermediate, rank = weights.shape
                if flat_intermediate % self.n_shared_experts != 0:
                    raise ValueError(
                        "Shared-sink gate/up LoRA-B height must be divisible by "
                        f"{self.n_shared_experts}, got {flat_intermediate}"
                    )
                weights = weights.view(
                    self.n_shared_experts,
                    flat_intermediate // self.n_shared_experts,
                    rank,
                )
        if self.moe_tp_size <= 1 or target_module != "gate_up_proj_moe":
            return weights
        if isinstance(weights, dict):
            return {
                expert_id: self._slice_gate_up_lora_b(weight, tp_rank)
                for expert_id, weight in weights.items()
            }
        if weights.dim() == 3:
            return torch.stack(
                [

View on GitHub (pinned to 0132848349)

Solutions

1. Confirm the LoRA-B height matches the model's flattened shared-expert intermediate width (divisible by n_shared_experts)
2. Re-export the adapter from the same base model config
3. For down-proj tensors, pass target_module='down_proj_moe' so they are unsqueezed rather than divisibility-checked

Example fix

# before
lora_b = torch.randn(3073, rank)  # not divisible by n_shared_experts=2
# after
lora_b = torch.randn(2 * (model.intermediate_size // 2), rank)

if lora_b.dim() == 2 and lora_b.shape[0] % model.n_shared_experts != 0:
    raise ValueError('bad LoRA-B height before load')

def is_valid_lora_b(w, n_shared):
    return w.dim() == 2 and w.shape[0] % n_shared == 0

When it happens

Trigger: Loading a gate/up LoRA-B tensor with flat_intermediate % n_shared_experts != 0 while target_module is not down_proj_moe.

Common situations: Adapter exported with a different intermediate size or expert grouping than the served Inkling model; wrong tensor passed to the B-slicing path.

Related errors

• Shared-sink down LoRA-A width must be divisible by {self.n_s
• Mixed shared-outer LoRA formats detected across loaded adapt
• {gate_name} has shape {tuple(g.shape)}, expected ({H * D}, {
• The output_size of gate's and up's weight = {intermediate_si
• The input_size of down's weight = {intermediate_size_per_par

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/13703f314522c1bc. Report an issue: GitHub.