huggingface/transformers · error · NotImplementedError

Sliding window attention layers do not support block table

Error message

Sliding window attention layers do not support block table

What it means

fill_block_table for sliding-window attention layers is intentionally unimplemented (marked TODO in cache_manager.py): building the paged-attention block table for SWA layers is not supported yet, so any code path that reaches it fails loudly.

Source

Thrown at src/transformers/generation/continuous_batching/cache_manager.py:548

        cache_length = min(query_length, self.sliding_window)
        padding_length = query_length - cache_length
        # Compute the physical indices
        physical_indices = []
        for i in range(start_index, start_index + cache_length):
            i %= self.sliding_window
            block_idx = i // self.block_size
            block_offset = i % self.block_size
            physical_index = block_table[block_idx] * self.block_size + block_offset
            physical_indices.append(physical_index)
        if padding_length > 0:
            physical_indices = [self.write_trash_index] * padding_length + physical_indices
        return physical_indices

    # TODO: implement this
    def fill_block_table(
        self, request_id: str, past_length: int, query_length: int, block_table: torch.Tensor
    ) -> None:
        raise NotImplementedError("Sliding window attention layers do not support block table")

View on GitHub (pinned to a597f97485)

Solutions

1. Use a model variant without sliding-window attention for continuous batching
2. Track the upstream issue/PR implementing SWA block tables and upgrade once merged
3. As a workaround, disable sliding window in the model config only if the architecture allows an equivalent full-attention variant

sw = getattr(model.config, 'sliding_window', None)
uses_swa = bool(getattr(model.config, 'use_sliding_window', False)) or sw is not None
if uses_swa:
    raise NotImplementedError('SWA models unsupported for continuous batching here — pick a full-attention variant')

try:
    manager = model.continuous_batching()
except NotImplementedError as e:
    if 'block table' in str(e):
        model = AutoModelForCausalLM.from_pretrained(full_attention_variant)
        manager = model.continuous_batching()
    else:
        raise

When it happens

Trigger: Running continuous batching with a model that has sliding-window attention layers (e.g. Gemma-2/Mistral-SW style) on a path that must fill a block table for the SWA layer group — typically long generations that exceed the window and require the block-table path rather than the fast path.

Common situations: Switching a working full-attention model to a sliding-window variant; upgrading transformers where SWA cache support is still landing; prompts longer than the sliding window under CB.

Related errors

• Invalid group type: {}
• The following layers have the mutually exclusive `sliding_wi
• This method should be implemented by the derived class.
• `crop` was called, but the current layer does not track past
• Once the sliding window size has been reached, `DynamicSlidi

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/e208cc13a97eb564. Report an issue: GitHub.