huggingface/transformers · error · ValueError

You called `get_mask_sizes` on layer index {layer_idx}, but

Error message

You called `get_mask_sizes` on layer index {layer_idx}, but this layer is a LinearAttention layer, which does not track sequence length.

What it means

Cache.get_mask_sizes() raises ValueError when called with an explicit non-zero layer_idx pointing at a LinearAttention layer. Mask sizes depend on KV sequence length, which linear attention layers do not track; only the default path (layer_idx=0, re-dispatched to the first attention layer) is valid.

Source

Thrown at src/transformers/cache_utils.py:1570

            return all(self.layers[layer_idx].has_previous_state.values())
        return self.layers[layer_idx].has_previous_state[state_idx]

    def get_mask_sizes(self, query_length: int, layer_idx: int) -> tuple[int, int]:
        """
        Return a tuple (kv_length, kv_offset) corresponding to the length and offset that will be returned for
        the given layer at `layer_idx`.
        The masks are then prepared according to the given lengths (kv_length, kv_offset) and patterns for each layer.
        """
        # For DynamicCache, where the layers are created at runtime -> if it was not yet created, the size is
        # simply the query_length
        if layer_idx >= len(self.layers):
            return query_length, 0

        # For alternating attention/linear attention caches, `get_mask_sizes` needs to use attention layer idx when called with default layer_idx
        if not isinstance(self.layers[layer_idx], CacheLayerMixin):
            # If this is called with non-default arg, raise
            if layer_idx != 0:
                raise ValueError(
                    f"You called `get_mask_sizes` on layer index {layer_idx}, but this layer is a LinearAttention layer, which "
                    "does not track sequence length."
                )
            try:
                # Use the first attention layer
                layer_idx = next(idx for idx in range(len(self)) if isinstance(self.layers[idx], CacheLayerMixin))
            except StopIteration:
                raise ValueError(
                    "`get_mask_sizes` can only be called on Attention layers, and the current Cache seem to only contain "
                    "LinearAttention layers."
                )

        return self.layers[layer_idx].get_mask_sizes(query_length)

    def get_query_offset(self, layer_idx: int = 0) -> int:
        """Returns the current offset of the query for the given `layer_idx`. It's always equal to the cache length, i.e.
        `get_seq_length(layer_idx)`, except for MTP layers.
        """

View on GitHub (pinned to a597f97485)

Solutions

1. Call with layer_idx=0 / default to auto-select the first attention layer, or pass an attention layer index
2. Skip linear attention indices: isinstance check on cache.layers[idx]
3. Rely on the model's built-in mask preparation, which dispatches correctly

Example fix

# before
for i in range(len(cache.layers)):
    kv_len, off = cache.get_mask_sizes(q_len, layer_idx=i)

# after
for i in range(len(cache.layers)):
    if isinstance(cache.layers[i], CacheLayerMixin):
        kv_len, off = cache.get_mask_sizes(q_len, layer_idx=i)

from transformers.cache_utils import CacheLayerMixin

if layer_idx == 0 or isinstance(cache.layers[layer_idx], CacheLayerMixin):
    kv_len, offset = cache.get_mask_sizes(query_length, layer_idx=layer_idx)

from transformers.cache_utils import CacheLayerMixin

def tracks_mask_sizes(layer) -> bool:
    return isinstance(layer, CacheLayerMixin)

When it happens

Trigger: cache.get_mask_sizes(query_length, layer_idx=k) with k != 0 where layers[k] is linear attention — typically inside per-layer mask preparation for hybrid models.

Common situations: Custom attention-mask preparation loops over all layers; alternating attention/linear attention architectures where most indices are linear attention.

Related errors

• Cannot call `update_conv_state` on a non-LinearAttention lay
• You called `get_seq_length` on layer index {layer_idx}, but
• You called `has_previous_state` on layer index {layer_idx},
• `get_mask_sizes` can only be called on Attention layers, and
• Cannot call `update_indexer` on layer {layer_idx} which is a

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/41325e986350d076. Report an issue: GitHub.