huggingface/transformers · error · ValueError

You called `get_seq_length` on layer index

Error message

You called `get_seq_length` on layer index {layer_idx}, but this layer is a LinearAttention layer, which does not track sequence length.

What it means

Cache.get_seq_length() raises ValueError when called with an explicit non-zero layer_idx pointing at a LinearAttention layer. Linear attention layers have no sequence-length dimension (fixed-size states), so only the automatic default path (layer_idx=0, which re-dispatches to the first attention layer) is allowed.

Solutions

1. Only query attention layer indices, or call get_seq_length() with the default layer_idx to auto-select the first attention layer
2. Filter indices by isinstance(layer, CacheLayerMixin) before querying
3. For hybrid caches, use config.layer_types to know which indices are safe

Example fix

# before
lengths = [cache.get_seq_length(i) for i in range(len(cache.layers))]

# after
lengths = [
    cache.get_seq_length(i)
    for i in range(len(cache.layers))
    if isinstance(cache.layers[i], CacheLayerMixin)
]

from transformers.cache_utils import CacheLayerMixin

if layer_idx != 0:
    assert isinstance(cache.layers[layer_idx], CacheLayerMixin), "not an attention layer"
length = cache.get_seq_length(layer_idx)

from transformers.cache_utils import CacheLayerMixin

def tracks_seq_length(layer) -> bool:
    return isinstance(layer, CacheLayerMixin)

When it happens

Trigger: cache.get_seq_length(layer_idx=k) with k != 0 where layers[k] is a linear attention layer — e.g. looping over all layer indices to collect per-layer lengths on a hybrid cache.

Common situations: Per-layer diagnostics or logging loops copied from pure-attention models; code that assumes every layer tracks KV length; alternating attention/linear attention models (e.g. Qwen3-Next, Falcon-H1) where many indices are linear attention.

Related errors

• Cannot call `update_conv_state` on a non-LinearAttention…
• `get_seq_length` can only be called on Attention layers…
• You called `get_mask_sizes` on layer index
• You called `has_previous_state` on layer index
• Cannot call `update_indexer` on layer

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/571a9813e21a56c8. Report an issue: GitHub.

Appendix: source

Thrown at src/transformers/cache_utils.py:1494

            if not layer.supports_early_init or layer.is_initialized:
                continue
            # Note that the initialization needs all dimensions (except -2), as well as device and dtype, so we use
            # this fake tensor approach. It has size 0 on the -2 dimension, so it does not allocate any data (it only
            # creates an empty tensor with correct shape, dtype and device), which is very efficient and practical
            fake_kv_tensor = torch.zeros((batch_size, layer_num_heads, 0, layer_head_dim), dtype=dtype, device=device)
            # Init the layer
            layer.lazy_initialization(fake_kv_tensor, fake_kv_tensor)

    def get_seq_length(self, layer_idx: int = 0) -> int:
        """Returns the sequence length of the cache for the given layer."""
        if layer_idx >= len(self.layers):
            return 0

        # For alternating attention/linear attention  caches, `get_seq_length` needs to use attention layer idx when called with default layer_idx
        if not isinstance(self.layers[layer_idx], CacheLayerMixin):
            # If this is called with non-default arg, raise
            if layer_idx != 0:
                raise ValueError(
                    f"You called `get_seq_length` on layer index {layer_idx}, but this layer is a LinearAttention layer, which "
                    "does not track sequence length."
                )
            try:
                # Use the first attention layer
                layer_idx = next(idx for idx in range(len(self)) if isinstance(self.layers[idx], CacheLayerMixin))
            except StopIteration:
                raise ValueError(
                    "`get_seq_length` can only be called on Attention layers, and the current Cache seem to only contain "
                    "LinearAttention layers."
                )

        return self.layers[layer_idx].get_seq_length()

    def get_max_length(self, layer_idx: int | None = None) -> int:
        """
        Returns the maximum length of the cache. If `layer_idx` is not provided (default), this returns the maximum
        across all layers. Otherwise, return the maximum supported value for the given layer.

View on GitHub (pinned to a597f97485)