sgl-project/sglang · error · ValueError

Cannot deinterleave odd gate/up dimension {dim}: {tuple(weig

Error message

Cannot deinterleave odd gate/up dimension {dim}: {tuple(weight.shape)}

What it means

deinterleave_gate_up converts Inkling's interleaved [gate0, up0, gate1, up1, ...] layout to the stock [gate..., up...] layout by splitting the given dimension in half; an odd-sized dimension cannot be split evenly, so it raises ValueError.

Source

Thrown at python/sglang/srt/models/inkling_common/util.py:70

    backend = get_moe_runner_backend()
    if lora_compatible_layout_enabled():
        return False
    return backend.is_flashinfer_trtllm_routed()


def trtllm_bf16_weight_prep_enabled() -> bool:
    """Return whether BF16 weights require TRT-LLM's ``[up || gate]`` layout."""
    from sglang.srt.layers.moe import get_moe_runner_backend

    backend = get_moe_runner_backend()
    return backend.is_flashinfer_trtllm() or backend.is_flashinfer_trtllm_routed()


def deinterleave_gate_up(weight: torch.Tensor, dim: int) -> torch.Tensor:
    """Convert Inkling [gate0, up0, ...] interleaved layout to stock [gate..., up...]."""
    dim = dim % weight.dim()
    if weight.shape[dim] % 2 != 0:
        raise ValueError(
            f"Cannot deinterleave odd gate/up dimension {dim}: {tuple(weight.shape)}"
        )
    shape = list(weight.shape)
    half = shape[dim] // 2
    view_shape = shape[:dim] + [half, 2] + shape[dim + 1 :]
    return (
        weight.reshape(view_shape)
        .transpose(dim, dim + 1)
        .reshape_as(weight)
        .contiguous()
    )


class FusedMoELoadingMixin(abc.ABC):
    def __init__(
        self,
        quant_config: QuantizationConfig | None,
        quant_method: UnquantizedFusedMoEMethod,

View on GitHub (pinned to 0132848349)

Solutions

1. Verify the tensor is actually the fused interleaved gate/up weight and dim is correct
2. Check the checkpoint's gate_up dimension equals 2 * intermediate_size
3. Re-export or re-shard the checkpoint so the fused dimension stays even

Example fix

# before
w = deinterleave_gate_up(gate_up_w, dim=1)  # shape[1] == 4097 -> raises
# after
assert gate_up_w.shape[1] % 2 == 0
w = deinterleave_gate_up(gate_up_w, dim=1)

assert weight.shape[dim] % 2 == 0, weight.shape

def is_deinterleavable(w: torch.Tensor, dim: int) -> bool:
    return w.shape[dim % w.dim()] % 2 == 0

When it happens

Trigger: Calling deinterleave_gate_up(weight, dim) where weight.shape[dim] is odd — e.g. a corrupt/sliced checkpoint tensor or wrong dim passed during load_weights.

Common situations: Checkpoint exported with non-interleaved or differently sharded gate/up weights; TP sharding slicing the fused dimension into an odd size; passing the wrong dim index.

Related errors

• {gate_name} has shape {tuple(g.shape)}, expected ({H * D}, {
• Shared-sink down LoRA-A width must be divisible by {self.n_s
• Shared-sink gate/up LoRA-B height must be divisible by {self
• Mobius fused gate/up destination is missing: {parameter_name
• qkv_proj weight {name}: unexpected shape {tuple(loaded_weigh

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6a26860493a1fb81. Report an issue: GitHub.