huggingface/transformers · error · ValueError

`get_mask_sizes` can only be called on Attention layers, and

Error message

`get_mask_sizes` can only be called on Attention layers, and the current Cache seem to only contain LinearAttention layers.

What it means

Cache.get_mask_sizes() raises ValueError (StopIteration fallback) when the default layer_idx points at a linear attention layer and the cache contains no attention layers at all. With no CacheLayerMixin layer there is no KV length to size masks against.

Source

Thrown at src/transformers/cache_utils.py:1578

        """
        # For DynamicCache, where the layers are created at runtime -> if it was not yet created, the size is
        # simply the query_length
        if layer_idx >= len(self.layers):
            return query_length, 0

        # For alternating attention/linear attention caches, `get_mask_sizes` needs to use attention layer idx when called with default layer_idx
        if not isinstance(self.layers[layer_idx], CacheLayerMixin):
            # If this is called with non-default arg, raise
            if layer_idx != 0:
                raise ValueError(
                    f"You called `get_mask_sizes` on layer index {layer_idx}, but this layer is a LinearAttention layer, which "
                    "does not track sequence length."
                )
            try:
                # Use the first attention layer
                layer_idx = next(idx for idx in range(len(self)) if isinstance(self.layers[idx], CacheLayerMixin))
            except StopIteration:
                raise ValueError(
                    "`get_mask_sizes` can only be called on Attention layers, and the current Cache seem to only contain "
                    "LinearAttention layers."
                )

        return self.layers[layer_idx].get_mask_sizes(query_length)

    def get_query_offset(self, layer_idx: int = 0) -> int:
        """Returns the current offset of the query for the given `layer_idx`. It's always equal to the cache length, i.e.
        `get_seq_length(layer_idx)`, except for MTP layers.
        """
        # It's simply equal to the length of the past states, except in very specific cases, see `MtpCache`
        return self.get_seq_length(layer_idx=layer_idx)

    def reset(self):
        """Recursively reset all layers tensors"""
        for layer_idx in range(len(self.layers)):
            self.layers[layer_idx].reset()

View on GitHub (pinned to a597f97485)

Solutions

1. Do not prepare attention masks for pure linear attention models — they are unused
2. Guard with any(isinstance(l, CacheLayerMixin) for l in cache.layers) before calling
3. Branch on model architecture (presence of layer_types == 'full_attention') in shared code

Example fix

# before
kv_len, offset = cache.get_mask_sizes(q_len)  # all-linear cache

# after
if any(isinstance(l, CacheLayerMixin) for l in cache.layers):
    kv_len, offset = cache.get_mask_sizes(q_len)
else:
    kv_len, offset = q_len, 0

from transformers.cache_utils import CacheLayerMixin

if any(isinstance(l, CacheLayerMixin) for l in cache.layers):
    kv_len, offset = cache.get_mask_sizes(query_length)
else:
    kv_len, offset = query_length, 0  # pure linear attention: no attention masks needed

from transformers.cache_utils import CacheLayerMixin

def needs_attention_masks(cache) -> bool:
    return any(isinstance(l, CacheLayerMixin) for l in cache.layers)

When it happens

Trigger: Calling get_mask_sizes() on a pure linear-attention cache (Mamba-style model with no attention layers).

Common situations: Generic generation utilities that prepare masks for every cache type; switching pipelines from hybrid to pure linear-attention models.

Related errors

• You called `get_mask_sizes` on layer index {layer_idx}, but
• Cannot call `update_conv_state` on a non-LinearAttention lay
• You called `get_seq_length` on layer index {layer_idx}, but
• `get_seq_length` can only be called on Attention layers, and
• `has_previous_state` can only be called on LinearAttention l

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/2bd820f2bd648210. Report an issue: GitHub.