{"record":{"id":"763858a357bf825e","repo":"hpcaitech/Open-Sora","slug":"the-last-dimension-d-must-be-even","errorCode":null,"errorMessage":"The last dimension D must be even.","messagePattern":"The last dimension D must be even\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"opensora/models/mmdit/math.py","lineNumber":81,"sourceCode":"    xq_out = freqs_cis[..., 0] * xq_[..., 0] + freqs_cis[..., 1] * xq_[..., 1]\n    xk_out = freqs_cis[..., 0] * xk_[..., 0] + freqs_cis[..., 1] * xk_[..., 1]\n    return xq_out.reshape(*xq.shape).type_as(xq), xk_out.reshape(*xk.shape).type_as(xk)\n\n\ndef rearrange_tensor(tensor):\n    \"\"\"\n    Rearranges the last dimension (D) of the input tensor based on the specified mapping:\n    2d -> d, 2d+1 -> D/2 + d.\n\n    Args:\n        tensor (torch.Tensor): Input tensor of shape [B, H, L, D], where D is even.\n\n    Returns:\n        torch.Tensor: Tensor with rearranged last dimension, same shape as input.\n    \"\"\"\n    B, H, L, D = tensor.shape\n    if D % 2 != 0:\n        raise ValueError(\"The last dimension D must be even.\")\n\n    half_D = D // 2\n    indices = torch.empty(D, dtype=torch.long, device=tensor.device)\n\n    # Fill the indices based on the mapping rule\n    indices[:half_D] = torch.arange(0, D, 2, device=tensor.device)\n    indices[half_D:] = torch.arange(1, D, 2, device=tensor.device)\n\n    # Rearrange the tensor based on the computed indices\n    return tensor.index_select(dim=-1, index=indices)\n\n\ndef reverse_rearrange_tensor(tensor):\n    \"\"\"\n    Restores the original order of the last dimension (D) of the input tensor based on the reverse mapping:\n    d -> 2d, D/2 + d -> 2d + 1.\n\n    Args:","sourceCodeStart":63,"sourceCodeEnd":99,"githubUrl":"https://github.com/hpcaitech/Open-Sora/blob/7ad6a96a135feb81f755c84fb391818718f6beb2/opensora/models/mmdit/math.py#L63-L99","documentation":"rearrange_tensor interleaves a 4D [B, H, L, D] tensor's head dimension by splitting D into two halves and shuffling even/odd indices (Rearrange '... (s d) -> ... d s' style used before interleaved QKV projection packing). This permutation only exists when D is even, so an odd last dimension raises immediately.","triggerScenarios":"Calling rearrange_tensor(tensor) where tensor.shape[-1] is odd — e.g. a projection output of size 3*head_dim per-head being fed with an incompatible head packing, or a custom attention head_dim producing an odd D.","commonSituations":"Changing num_heads/hidden_size so that per-head dim becomes odd; feeding attention weights that were packed for interleaved layouts into a tensor whose last dim is not 2-divisible.","solutions":["Check tensor.shape[-1] % 2 == 0 before calling; if odd, your upstream projection or reshape is wrong","Verify num_heads divides hidden_size and the per-head dimension layout matches what rearrange_tensor expects","Use the matching reverse_rearrange_tensor after the operation to round-trip correctly"],"exampleFix":"# before\ny = rearrange_tensor(x)  # x.shape[-1] == 375\n# after\nassert x.shape[-1] % 2 == 0, f\"odd head dim {x.shape[-1]}\"\ny = rearrange_tensor(x)","handlingStrategy":"validation","validationCode":"assert tensor.dim() == 4 and tensor.shape[-1] % 2 == 0, f\"need even last dim, got {tensor.shape}\"","typeGuard":"def has_even_head_dim(t: torch.Tensor) -> bool:\n    return t.dim() == 4 and t.shape[-1] % 2 == 0","tryCatchPattern":null,"preventionTips":["Assert even head dims in attention weight-loading utilities","Keep num_heads * 2-compatible head dims in configs","Round-trip test rearrange/reverse_rearrange on your shapes"],"tags":["pytorch","attention","tensor-shape","mmdit"],"backgroundTag":"invalid-tensor-shape","analyzedSha":"7ad6a96a135feb81f755c84fb391818718f6beb2","analyzedAt":"2026-08-28T16:58:37.171Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}