{"record":{"id":"37ea5983351c3f08","repo":"sgl-project/sglang","slug":"head-dim-must-be-a-multiple-of-8-got-head-dim","errorCode":null,"errorMessage":"head_dim must be a multiple of 8, got {head_dim}.","messagePattern":"head_dim must be a multiple of 8, got (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/decoders/ltx_2_5_diffusion_decoder.py","lineNumber":236,"sourceCode":"        _compile=True,\n    )\n    if len(_BLOCK_MASK_CACHE) >= _BLOCK_MASK_CACHE_MAX:\n        _BLOCK_MASK_CACHE.pop(next(iter(_BLOCK_MASK_CACHE)))\n    _BLOCK_MASK_CACHE[cache_key] = block_mask\n    return block_mask\n\n\nclass LTX2VideoVaeRotaryPosEmbed3D(nn.Module):\n    \"\"\"Absolute 3D rotary embedding over the (T, H, W) grid.\n\n    `head_dim` splits into (T, H, W) chunks, each rotated by its own axis\n    position.\n    \"\"\"\n\n    def __init__(self, head_dim: int, base: float = 10000.0) -> None:\n        super().__init__()\n        if head_dim % 8 != 0:\n            raise ValueError(f\"head_dim must be a multiple of 8, got {head_dim}.\")\n        # A quarter to T, the rest split H/W, both kept even for whole\n        # rotation pairs.\n        dim_t = (head_dim // 4) // 2 * 2\n        dim_hw = (head_dim - dim_t) // 2\n        if dim_hw % 2 != 0:\n            dim_t -= 2\n            dim_hw = (head_dim - dim_t) // 2\n        self.rope_dim_split = (dim_t, dim_hw, dim_hw)\n        self.base = base\n\n    def _axis_tables(\n        self, length: int, dim: int, device: torch.device\n    ) -> tuple[torch.Tensor, torch.Tensor]:\n        exponents = torch.arange(0, dim, 2, dtype=torch.float64, device=device) / dim\n        inv_freqs = (1.0 / self.base**exponents).to(torch.float32)\n        positions = torch.arange(length, dtype=torch.float32, device=device)\n        angles = positions[:, None] * inv_freqs[None, :]\n        return angles.cos(), angles.sin()","sourceCodeStart":218,"sourceCodeEnd":254,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/decoders/ltx_2_5_diffusion_decoder.py#L218-L254","documentation":"The 3D RoPE embeddings in the LTX-2.5 diffusion decoder split head_dim into temporal and spatial rotation pairs; a head_dim not divisible by 8 cannot be split into whole even-sized halves, so init fails fast with this check.","triggerScenarios":"Constructing the RoPE module with head_dim values like 12, 20, 48+4, or any non-multiple of 8 (e.g. head_dim=50), typically via a custom arch config overriding decoder head_dim.","commonSituations":"Porting a model whose attention head_dim is unusual; hand-editing architecture hyperparameters; experimenting with smaller head dims to shrink parameters.","solutions":["Use a head_dim that is a multiple of 8 (64, 128, 32, ...)","Keep the model's shipped default head_dim (usually 64) unless you have re-derived compatible dims","Validate arch head_dim at config-load time before constructing the decoder"],"exampleFix":"# before\nrope = LtxRotaryEmbedding3D(head_dim=50)\n# after\nrope = LtxRotaryEmbedding3D(head_dim=48)  # or 64","handlingStrategy":"validation","validationCode":"if head_dim % 8 != 0:\n    raise ValueError(f\"head_dim {head_dim} must be a multiple of 8\")","typeGuard":"def valid_head_dim(head_dim: int) -> bool:\n    return head_dim > 0 and head_dim % 8 == 0","tryCatchPattern":"try:\n    rope = LtxRotaryEmbedding3D(head_dim=head_dim)\nexcept ValueError:\n    head_dim = (head_dim // 8) * 8 or 8\n    rope = LtxRotaryEmbedding3D(head_dim=head_dim)","preventionTips":["Keep head_dim in {8, 16, 32, 64, 128}","Validate arch overrides in one place before constructing decoder modules"],"tags":["rope","model-config","shape-validation","ltx-2"],"backgroundTag":"invalid-head-dimension","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}