{"record":{"id":"79184a9cf0de1fbf","repo":"sgl-project/sglang","slug":"neighborhood-attention-requires-each-dim-to-be-at","errorCode":null,"errorMessage":"Neighborhood attention requires each dim to be at least its kernel size; got (T, H, W) = ({num_frames}, {height}, {width}) with kernel_size {self.kernel_size}.","messagePattern":"Neighborhood attention requires each dim to be at least its kernel size; got \\(T, H, W\\) = \\((.+?), (.+?), (.+?)\\) with kernel_size (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/decoders/ltx_2_5_diffusion_decoder.py","lineNumber":424,"sourceCode":"        return query, key, value\n\n    def build_block_mask(self, hidden_states: torch.Tensor):\n        \"\"\"The window mask for this grid, or `None` when NATTEN handles it.\n\n        Fixed within a stage, so built once.\n        \"\"\"\n        if _na3d() is not None:\n            return None\n        num_frames, height, width = hidden_states.shape[1:4]\n        return _neighborhood_block_mask(\n            num_frames, height, width, self.kernel_size, hidden_states.device\n        )\n\n    def forward(self, hidden_states: torch.Tensor, block_mask=None) -> torch.Tensor:\n        batch_size, num_frames, height, width, _ = hidden_states.shape\n        kernel_t, kernel_h, kernel_w = self.kernel_size\n        if num_frames < kernel_t or height < kernel_h or width < kernel_w:\n            raise ValueError(\n                \"Neighborhood attention requires each dim to be at least its \"\n                f\"kernel size; got (T, H, W) = ({num_frames}, {height}, {width}) \"\n                f\"with kernel_size {self.kernel_size}.\"\n            )\n\n        query, key, value = self.project_qkv(hidden_states)\n\n        na3d = _na3d()\n        if na3d is not None:\n            # `project_qkv` already yields NATTEN's layout. scale=1.0: the\n            # query is pre-scaled there.\n            hidden_states = na3d(\n                query, key, value, kernel_size=self.kernel_size, scale=1.0\n            )\n            hidden_states = hidden_states.reshape(\n                batch_size, num_frames, height, width, self.heads * self.head_dim\n            )\n            return self.to_out[0](hidden_states)","sourceCodeStart":406,"sourceCodeEnd":442,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/decoders/ltx_2_5_diffusion_decoder.py#L406-L442","documentation":"Neighborhood (windowed) attention needs every spatial/temporal dimension to be at least as large as its attention kernel so each position has a full neighborhood. The block checks (T,H,W) of the incoming hidden_states against kernel_size at forward time and raises when any dim is smaller than its kernel extent.","triggerScenarios":"Calling block.forward(hidden_states) where hidden_states has shape (B, T, H, W, C) with, e.g., T < kernel_t (fewer frames than the temporal kernel), or H/W smaller than the spatial kernel — common with tiny test videos, thumbnails, or heavily downsampled latents.","commonSituations":"Generating a 1-frame or very short clip with a (3,3,3)-kernel decoder; small resolutions like 32x32 latents after patching; unit tests using minimal dummy tensors; user requests for tiny aspect ratios.","solutions":["Increase resolution/frame count so T>=kernel_t, H>=kernel_h, W>=kernel_w (e.g. at least 3 frames and 3x3 latent grid for kernel (3,3,3))","If small inputs must be supported, pad the latent grid and crop after decoding","Use a decoder config with smaller kernel_size for small-input workloads","Validate requested resolution/frames against the decoder kernel before running the pipeline"],"exampleFix":"# before\nout = block(torch.randn(1, 1, 2, 2, C))  # T=1 < kernel_t=3\n# after\nout = block(torch.randn(1, 3, 4, 4, C))  # all dims >= kernel extents","handlingStrategy":"validation","validationCode":"kt, kh, kw = block.kernel_size\nT, H, W = num_frames, height, width\nif T < kt or H < kh or W < kw:\n    raise ValueError(f\"input (T,H,W)=({T},{H},{W}) smaller than kernel {block.kernel_size}; \"\n                     \"increase resolution/frames or pad\")","typeGuard":"def fits_kernel(t: int, h: int, w: int, kernel: tuple[int,int,int]) -> bool:\n    kt, kh, kw = kernel\n    return t >= kt and h >= kh and w >= kw","tryCatchPattern":"try:\n    out = block(hidden_states)\nexcept ValueError as e:\n    if \"kernel size\" in str(e):\n        pad = (max(0,kt-T), max(0,kh-H), max(0,kw-W))  # F.pad latent then crop output\n        raise\n    raise","preventionTips":["Enforce minimum resolution and frame count in the generation API (map user-facing sizes to latent dims first)","Pad short clips to kernel extents before decoding and crop afterwards"],"tags":["attention","input-shape","video-generation","ltx-2"],"backgroundTag":"input-smaller-than-kernel","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}