huggingface/transformers · error · ValueError

`get_mask_sizes` can only be called on Attention layers…

Error message

`get_mask_sizes` can only be called on Attention layers, and the current Cache seem to only contain LinearAttention layers.

What it means

Cache.get_mask_sizes() raises ValueError (StopIteration fallback) when the default layer_idx points at a linear attention layer and the cache contains no attention layers at all. With no CacheLayerMixin layer there is no KV length to size masks against.

Solutions

  1. Do not prepare attention masks for pure linear attention models — they are unused
  2. Guard with any(isinstance(l, CacheLayerMixin) for l in cache.layers) before calling
  3. Branch on model architecture (presence of layer_types == 'full_attention') in shared code

Example fix

# before
kv_len, offset = cache.get_mask_sizes(q_len)  # all-linear cache

# after
if any(isinstance(l, CacheLayerMixin) for l in cache.layers):
    kv_len, offset = cache.get_mask_sizes(q_len)
else:
    kv_len, offset = q_len, 0
Defensive patterns

Strategy: type-guard

Validate before calling

from transformers.cache_utils import CacheLayerMixin

if any(isinstance(l, CacheLayerMixin) for l in cache.layers):
    kv_len, offset = cache.get_mask_sizes(query_length)
else:
    kv_len, offset = query_length, 0  # pure linear attention: no attention masks needed

Type guard

from transformers.cache_utils import CacheLayerMixin

def needs_attention_masks(cache) -> bool:
    return any(isinstance(l, CacheLayerMixin) for l in cache.layers)

Prevention

When it happens

Trigger: Calling get_mask_sizes() on a pure linear-attention cache (Mamba-style model with no attention layers).

Common situations: Generic generation utilities that prepare masks for every cache type; switching pipelines from hybrid to pure linear-attention models.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/2bd820f2bd648210. Report an issue: GitHub.

Appendix: source

Thrown at src/transformers/cache_utils.py:1578

        """
        # For DynamicCache, where the layers are created at runtime -> if it was not yet created, the size is
        # simply the query_length
        if layer_idx >= len(self.layers):
            return query_length, 0

        # For alternating attention/linear attention caches, `get_mask_sizes` needs to use attention layer idx when called with default layer_idx
        if not isinstance(self.layers[layer_idx], CacheLayerMixin):
            # If this is called with non-default arg, raise
            if layer_idx != 0:
                raise ValueError(
                    f"You called `get_mask_sizes` on layer index {layer_idx}, but this layer is a LinearAttention layer, which "
                    "does not track sequence length."
                )
            try:
                # Use the first attention layer
                layer_idx = next(idx for idx in range(len(self)) if isinstance(self.layers[idx], CacheLayerMixin))
            except StopIteration:
                raise ValueError(
                    "`get_mask_sizes` can only be called on Attention layers, and the current Cache seem to only contain "
                    "LinearAttention layers."
                )

        return self.layers[layer_idx].get_mask_sizes(query_length)

    def get_query_offset(self, layer_idx: int = 0) -> int:
        """Returns the current offset of the query for the given `layer_idx`. It's always equal to the cache length, i.e.
        `get_seq_length(layer_idx)`, except for MTP layers.
        """
        # It's simply equal to the length of the past states, except in very specific cases, see `MtpCache`
        return self.get_seq_length(layer_idx=layer_idx)

    def reset(self):
        """Recursively reset all layers tensors"""
        for layer_idx in range(len(self.layers)):
            self.layers[layer_idx].reset()

View on GitHub (pinned to a597f97485)