{"record":{"id":"6a26860493a1fb81","repo":"sgl-project/sglang","slug":"cannot-deinterleave-odd-gate-up-dimension-dim","errorCode":null,"errorMessage":"Cannot deinterleave odd gate/up dimension {dim}: {tuple(weight.shape)}","messagePattern":"Cannot deinterleave odd gate/up dimension (.+?): (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/inkling_common/util.py","lineNumber":70,"sourceCode":"    backend = get_moe_runner_backend()\n    if lora_compatible_layout_enabled():\n        return False\n    return backend.is_flashinfer_trtllm_routed()\n\n\ndef trtllm_bf16_weight_prep_enabled() -> bool:\n    \"\"\"Return whether BF16 weights require TRT-LLM's ``[up || gate]`` layout.\"\"\"\n    from sglang.srt.layers.moe import get_moe_runner_backend\n\n    backend = get_moe_runner_backend()\n    return backend.is_flashinfer_trtllm() or backend.is_flashinfer_trtllm_routed()\n\n\ndef deinterleave_gate_up(weight: torch.Tensor, dim: int) -> torch.Tensor:\n    \"\"\"Convert Inkling [gate0, up0, ...] interleaved layout to stock [gate..., up...].\"\"\"\n    dim = dim % weight.dim()\n    if weight.shape[dim] % 2 != 0:\n        raise ValueError(\n            f\"Cannot deinterleave odd gate/up dimension {dim}: {tuple(weight.shape)}\"\n        )\n    shape = list(weight.shape)\n    half = shape[dim] // 2\n    view_shape = shape[:dim] + [half, 2] + shape[dim + 1 :]\n    return (\n        weight.reshape(view_shape)\n        .transpose(dim, dim + 1)\n        .reshape_as(weight)\n        .contiguous()\n    )\n\n\nclass FusedMoELoadingMixin(abc.ABC):\n    def __init__(\n        self,\n        quant_config: QuantizationConfig | None,\n        quant_method: UnquantizedFusedMoEMethod,","sourceCodeStart":52,"sourceCodeEnd":88,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/inkling_common/util.py#L52-L88","documentation":"deinterleave_gate_up converts Inkling's interleaved [gate0, up0, gate1, up1, ...] layout to the stock [gate..., up...] layout by splitting the given dimension in half; an odd-sized dimension cannot be split evenly, so it raises ValueError.","triggerScenarios":"Calling deinterleave_gate_up(weight, dim) where weight.shape[dim] is odd — e.g. a corrupt/sliced checkpoint tensor or wrong dim passed during load_weights.","commonSituations":"Checkpoint exported with non-interleaved or differently sharded gate/up weights; TP sharding slicing the fused dimension into an odd size; passing the wrong dim index.","solutions":["Verify the tensor is actually the fused interleaved gate/up weight and dim is correct","Check the checkpoint's gate_up dimension equals 2 * intermediate_size","Re-export or re-shard the checkpoint so the fused dimension stays even"],"exampleFix":"# before\nw = deinterleave_gate_up(gate_up_w, dim=1)  # shape[1] == 4097 -> raises\n# after\nassert gate_up_w.shape[1] % 2 == 0\nw = deinterleave_gate_up(gate_up_w, dim=1)","handlingStrategy":"validation","validationCode":"assert weight.shape[dim] % 2 == 0, weight.shape","typeGuard":"def is_deinterleavable(w: torch.Tensor, dim: int) -> bool:\n    return w.shape[dim % w.dim()] % 2 == 0","tryCatchPattern":null,"preventionTips":["Sanity-check fused gate/up dims (== 2*intermediate_size) when validating checkpoints"],"tags":["weight-loading","layout-conversion","shape-mismatch","interns2"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T11:17:15.048Z"}