huggingface/transformers · error · ValueError

You called `get_seq_length` on layer index {layer_idx}, but

Error message

You called `get_seq_length` on layer index {layer_idx}, but this layer is a LinearAttention layer, which does not track sequence length.

What it means

Cache.get_seq_length() raises ValueError when called with an explicit non-zero layer_idx pointing at a LinearAttention layer. Linear attention layers have no sequence-length dimension (fixed-size states), so only the automatic default path (layer_idx=0, which re-dispatches to the first attention layer) is allowed.

Source

Thrown at src/transformers/cache_utils.py:1494

            if not layer.supports_early_init or layer.is_initialized:
                continue
            # Note that the initialization needs all dimensions (except -2), as well as device and dtype, so we use
            # this fake tensor approach. It has size 0 on the -2 dimension, so it does not allocate any data (it only
            # creates an empty tensor with correct shape, dtype and device), which is very efficient and practical
            fake_kv_tensor = torch.zeros((batch_size, layer_num_heads, 0, layer_head_dim), dtype=dtype, device=device)
            # Init the layer
            layer.lazy_initialization(fake_kv_tensor, fake_kv_tensor)

    def get_seq_length(self, layer_idx: int = 0) -> int:
        """Returns the sequence length of the cache for the given layer."""
        if layer_idx >= len(self.layers):
            return 0

        # For alternating attention/linear attention  caches, `get_seq_length` needs to use attention layer idx when called with default layer_idx
        if not isinstance(self.layers[layer_idx], CacheLayerMixin):
            # If this is called with non-default arg, raise
            if layer_idx != 0:
                raise ValueError(
                    f"You called `get_seq_length` on layer index {layer_idx}, but this layer is a LinearAttention layer, which "
                    "does not track sequence length."
                )
            try:
                # Use the first attention layer
                layer_idx = next(idx for idx in range(len(self)) if isinstance(self.layers[idx], CacheLayerMixin))
            except StopIteration:
                raise ValueError(
                    "`get_seq_length` can only be called on Attention layers, and the current Cache seem to only contain "
                    "LinearAttention layers."
                )

        return self.layers[layer_idx].get_seq_length()

    def get_max_length(self, layer_idx: int | None = None) -> int:
        """
        Returns the maximum length of the cache. If `layer_idx` is not provided (default), this returns the maximum
        across all layers. Otherwise, return the maximum supported value for the given layer.

View on GitHub (pinned to a597f97485)

Solutions

1. Only query attention layer indices, or call get_seq_length() with the default layer_idx to auto-select the first attention layer
2. Filter indices by isinstance(layer, CacheLayerMixin) before querying
3. For hybrid caches, use config.layer_types to know which indices are safe

Example fix

# before
lengths = [cache.get_seq_length(i) for i in range(len(cache.layers))]

# after
lengths = [
    cache.get_seq_length(i)
    for i in range(len(cache.layers))
    if isinstance(cache.layers[i], CacheLayerMixin)
]

from transformers.cache_utils import CacheLayerMixin

if layer_idx != 0:
    assert isinstance(cache.layers[layer_idx], CacheLayerMixin), "not an attention layer"
length = cache.get_seq_length(layer_idx)

from transformers.cache_utils import CacheLayerMixin

def tracks_seq_length(layer) -> bool:
    return isinstance(layer, CacheLayerMixin)

When it happens

Trigger: cache.get_seq_length(layer_idx=k) with k != 0 where layers[k] is a linear attention layer — e.g. looping over all layer indices to collect per-layer lengths on a hybrid cache.

Common situations: Per-layer diagnostics or logging loops copied from pure-attention models; code that assumes every layer tracks KV length; alternating attention/linear attention models (e.g. Qwen3-Next, Falcon-H1) where many indices are linear attention.

Related errors

• Cannot call `update_conv_state` on a non-LinearAttention lay
• `get_seq_length` can only be called on Attention layers, and
• You called `has_previous_state` on layer index {layer_idx},
• You called `get_mask_sizes` on layer index {layer_idx}, but
• Cannot call `update_indexer` on layer {layer_idx} which is a

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/571a9813e21a56c8. Report an issue: GitHub.