{"record":{"id":"056e011ce79fe23f","repo":"sgl-project/sglang","slug":"dim-dim-must-be-divisible-by-head-dim-head-dim","errorCode":null,"errorMessage":"dim {dim} must be divisible by head_dim {head_dim}.","messagePattern":"dim (.+?) must be divisible by head_dim (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/decoders/ltx_2_5_diffusion_decoder.py","lineNumber":374,"sourceCode":"                        \"platform; falling back to eager\"\n                    ),\n                )\n        return self._apply_rope(query, tables), self._apply_rope(key, tables)\n\n\nclass LTX2VideoVaeNeighborhoodAttention(nn.Module):\n    \"\"\"3D neighborhood attention over a channels-last `(B, T, H, W, C)` volume.\"\"\"\n\n    def __init__(\n        self,\n        dim: int,\n        kernel_size: tuple[int, int, int],\n        head_dim: int = 64,\n        rope_base: float = 10000.0,\n    ) -> None:\n        super().__init__()\n        if dim % head_dim != 0:\n            raise ValueError(f\"dim {dim} must be divisible by head_dim {head_dim}.\")\n        self.heads = dim // head_dim\n        self.head_dim = head_dim\n        self.kernel_size = tuple(kernel_size)\n        self.scale = head_dim**-0.5\n\n        self.to_q = nn.Linear(dim, dim, bias=True)\n        self.to_k = nn.Linear(dim, dim, bias=True)\n        self.to_v = nn.Linear(dim, dim, bias=True)\n        self.to_out = nn.ModuleList([nn.Linear(dim, dim, bias=True), nn.Dropout(0.0)])\n        self.norm_q = nn.RMSNorm(head_dim, eps=1e-6)\n        self.norm_k = nn.RMSNorm(head_dim, eps=1e-6)\n        self.rope = LTX2VideoVaeRotaryPosEmbed3D(head_dim, base=rope_base)\n\n    def project_qkv(\n        self, hidden_states: torch.Tensor\n    ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:\n        \"\"\"Q/K/V as `(B, T, H, W, heads, head_dim)`: normed, query pre-scaled, rotated.\n","sourceCodeStart":356,"sourceCodeEnd":392,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/decoders/ltx_2_5_diffusion_decoder.py#L356-L392","documentation":"The decoder's neighborhood-attention block splits `dim` into `heads = dim // head_dim` attention heads; if dim is not an exact multiple of head_dim the reshape is impossible, so __init__ validates it up front.","triggerScenarios":"Constructing the attention block with dim not divisible by head_dim — e.g. dim=1000 with head_dim=64, or a stage_channels value from arch config that doesn't align with the default head_dim=64.","commonSituations":"Customizing decoder stage widths without adjusting head_dim; mixing config values from a different model revision where dims changed; typos in width lists.","solutions":["Align stage channel dims to be multiples of head_dim (e.g. multiples of 64)","Or pass head_dim that divides dim exactly (e.g. head_dim=50 for dim=1000)","Re-check the full decoder_stage_channels list against the shipped arch defaults"],"exampleFix":"# before\nblock = NeighborhoodAttentionBlock(dim=1000, kernel_size=(3,3,3), head_dim=64)\n# after\nblock = NeighborhoodAttentionBlock(dim=1024, kernel_size=(3,3,3), head_dim=64)","handlingStrategy":"validation","validationCode":"if dim % head_dim != 0:\n    raise ValueError(f\"dim {dim} not divisible by head_dim {head_dim}\")","typeGuard":"def dims_aligned(dim: int, head_dim: int) -> bool:\n    return head_dim > 0 and dim % head_dim == 0","tryCatchPattern":"try:\n    block = NeighborhoodAttentionBlock(dim=dim, kernel_size=k, head_dim=head_dim)\nexcept ValueError:\n    head_dim = next(h for h in (64, 32, 16, 8) if dim % h == 0)\n    block = NeighborhoodAttentionBlock(dim=dim, kernel_size=k, head_dim=head_dim)","preventionTips":["Use power-of-two channel widths that are multiples of your head_dim","Derive head_dim from dim programmatically in config builders"],"tags":["attention","model-config","divisibility","ltx-2"],"backgroundTag":"dim-not-divisible-by-head-dim","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}