{"record":{"id":"e208cc13a97eb564","repo":"huggingface/transformers","slug":"sliding-window-attention-layers-do-not-support-blo","errorCode":null,"errorMessage":"Sliding window attention layers do not support block table","messagePattern":"Sliding window attention layers do not support block table","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/continuous_batching/cache_manager.py","lineNumber":548,"sourceCode":"        cache_length = min(query_length, self.sliding_window)\n        padding_length = query_length - cache_length\n        # Compute the physical indices\n        physical_indices = []\n        for i in range(start_index, start_index + cache_length):\n            i %= self.sliding_window\n            block_idx = i // self.block_size\n            block_offset = i % self.block_size\n            physical_index = block_table[block_idx] * self.block_size + block_offset\n            physical_indices.append(physical_index)\n        if padding_length > 0:\n            physical_indices = [self.write_trash_index] * padding_length + physical_indices\n        return physical_indices\n\n    # TODO: implement this\n    def fill_block_table(\n        self, request_id: str, past_length: int, query_length: int, block_table: torch.Tensor\n    ) -> None:\n        raise NotImplementedError(\"Sliding window attention layers do not support block table\")\n","sourceCodeStart":530,"sourceCodeEnd":549,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/continuous_batching/cache_manager.py#L530-L549","documentation":"fill_block_table for sliding-window attention layers is intentionally unimplemented (marked TODO in cache_manager.py): building the paged-attention block table for SWA layers is not supported yet, so any code path that reaches it fails loudly.","triggerScenarios":"Running continuous batching with a model that has sliding-window attention layers (e.g. Gemma-2/Mistral-SW style) on a path that must fill a block table for the SWA layer group — typically long generations that exceed the window and require the block-table path rather than the fast path.","commonSituations":"Switching a working full-attention model to a sliding-window variant; upgrading transformers where SWA cache support is still landing; prompts longer than the sliding window under CB.","solutions":["Use a model variant without sliding-window attention for continuous batching","Track the upstream issue/PR implementing SWA block tables and upgrade once merged","As a workaround, disable sliding window in the model config only if the architecture allows an equivalent full-attention variant"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"sw = getattr(model.config, 'sliding_window', None)\nuses_swa = bool(getattr(model.config, 'use_sliding_window', False)) or sw is not None\nif uses_swa:\n    raise NotImplementedError('SWA models unsupported for continuous batching here — pick a full-attention variant')","typeGuard":null,"tryCatchPattern":"try:\n    manager = model.continuous_batching()\nexcept NotImplementedError as e:\n    if 'block table' in str(e):\n        model = AutoModelForCausalLM.from_pretrained(full_attention_variant)\n        manager = model.continuous_batching()\n    else:\n        raise","preventionTips":["Check config.sliding_window / use_sliding_window before CB inference","Track upstream SWA support","Keep a full-attention fallback checkpoint for serving"],"tags":["not-implemented","sliding-window","attention","continuous-batching"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}