huggingface/transformers · error · ValueError

`get_seq_length` can only be called on Attention layers, and

Error message

`get_seq_length` can only be called on Attention layers, and the current Cache seem to only contain LinearAttention layers.

What it means

Cache.get_seq_length() raises ValueError (via StopIteration fallback) when the default layer_idx=0 lands on a linear attention layer and NO layer in the cache is an attention layer. The method searches for the first CacheLayerMixin; on an all-linear-attention cache there is nothing to measure sequence length on.

Source

Thrown at src/transformers/cache_utils.py:1502

    def get_seq_length(self, layer_idx: int = 0) -> int:
        """Returns the sequence length of the cache for the given layer."""
        if layer_idx >= len(self.layers):
            return 0

        # For alternating attention/linear attention  caches, `get_seq_length` needs to use attention layer idx when called with default layer_idx
        if not isinstance(self.layers[layer_idx], CacheLayerMixin):
            # If this is called with non-default arg, raise
            if layer_idx != 0:
                raise ValueError(
                    f"You called `get_seq_length` on layer index {layer_idx}, but this layer is a LinearAttention layer, which "
                    "does not track sequence length."
                )
            try:
                # Use the first attention layer
                layer_idx = next(idx for idx in range(len(self)) if isinstance(self.layers[idx], CacheLayerMixin))
            except StopIteration:
                raise ValueError(
                    "`get_seq_length` can only be called on Attention layers, and the current Cache seem to only contain "
                    "LinearAttention layers."
                )

        return self.layers[layer_idx].get_seq_length()

    def get_max_length(self, layer_idx: int | None = None) -> int:
        """
        Returns the maximum length of the cache. If `layer_idx` is not provided (default), this returns the maximum
        across all layers. Otherwise, return the maximum supported value for the given layer.
        A value of `-1` means no maximum, or undefined maximum, e.g. for dynamic attention layers that can grow indefinitely,
        or linear attention layer that do not have a sequence length dimension.
        """
        # For DynamicCache, where the layers are created at runtime
        if layer_idx is not None and layer_idx >= len(self.layers):
            return -1

        if layer_idx is None:

View on GitHub (pinned to a597f97485)

Solutions

1. Skip sequence-length queries for pure linear attention models — their caches have no seq dimension
2. Guard with any(isinstance(l, CacheLayerMixin) for l in cache.layers) before calling
3. Use model-specific state handling (has_previous_state) for linear-attention models instead

Example fix

# before
past_len = cache.get_seq_length()  # pure linear attention cache

# after
if any(isinstance(l, CacheLayerMixin) for l in cache.layers):
    past_len = cache.get_seq_length()
else:
    past_len = None  # linear attention cache: no sequence length

from transformers.cache_utils import CacheLayerMixin

if any(isinstance(l, CacheLayerMixin) for l in cache.layers):
    seq_len = cache.get_seq_length()
else:
    seq_len = None  # pure linear attention cache: no sequence length

from transformers.cache_utils import CacheLayerMixin

def cache_has_attention_layers(cache) -> bool:
    return any(isinstance(l, CacheLayerMixin) for l in cache.layers)

When it happens

Trigger: Calling get_seq_length() on a cache built exclusively from LinearAttentionCacheLayerMixin layers (pure Mamba/linear-attention models).

Common situations: Generic generation or logging code that unconditionally calls cache.get_seq_length() on any cache; switching a pipeline from a hybrid model to a pure linear-attention model.

Related errors

• You called `get_seq_length` on layer index {layer_idx}, but
• Cannot call `update_conv_state` on a non-LinearAttention lay
• `has_previous_state` can only be called on LinearAttention l
• You called `has_previous_state` on layer index {layer_idx},
• You called `get_mask_sizes` on layer index {layer_idx}, but

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/cbb72156ff1f7f69. Report an issue: GitHub.