huggingface/transformers · error · ValueError

Cannot call `update_conv_state` on a non-LinearAttention lay

Error message

Cannot call `update_conv_state` on a non-LinearAttention layer!

What it means

Cache.update_conv_state() raises ValueError when the target layer is not a LinearAttentionCacheLayerMixin. Conv states exist only on linear attention layers (convolutional prefill states in Mamba/linear-attention hybrids); calling the conv-state API on an attention layer is a category error caught by an isinstance check.

Source

Thrown at src/transformers/cache_utils.py:1401

    def update_conv_state(
        self, conv_states: torch.Tensor, layer_idx: int, state_idx: int = 0, **kwargs
    ) -> torch.Tensor:
        """
        Updates the cache with the new `conv_states` for the layer `layer_idx`.

        Parameters:
            conv_states (`torch.Tensor`):
                The new conv states to cache.
            layer_idx (`int`):
                The index of the layer to cache the states for.

        Return:
            `torch.Tensor`: The updated conv states.
        """
        # NOTE: if we slightly break `update` arg order, we could combine this with it, and allow offloading support
        # out of the box
        if not isinstance(self.layers[layer_idx], LinearAttentionCacheLayerMixin):
            raise ValueError("Cannot call `update_conv_state` on a non-LinearAttention layer!")
        conv_states = self.layers[layer_idx].update_conv_state(conv_states, state_idx, **kwargs)
        return conv_states

    def update_recurrent_state(
        self, recurrent_states: torch.Tensor, layer_idx: int, state_idx: int = 0, **kwargs
    ) -> torch.Tensor:
        """
        Updates the cache with the new `recurrent_states` for the layer `layer_idx`.

        Parameters:
            smm_states (`torch.Tensor`):
                The new ssm states to cache.
            layer_idx (`int`):
                The index of the layer to cache the states for.

        Return:
            `torch.Tensor`: The updated ssm states.
        """

View on GitHub (pinned to a597f97485)

Solutions

1. Only call update_conv_state for layer indices that correspond to linear attention layers (check config.layer_types)
2. Let the model's forward call these APIs itself rather than driving cache updates manually
3. Verify the layer type at runtime: isinstance(cache.layers[idx], LinearAttentionCacheLayerMixin)

Example fix

# before
cache.update_conv_state(conv_states, layer_idx=0)  # layer 0 is full_attention

# after
linear_idx = next(i for i, t in enumerate(config.layer_types) if t == "linear_attention")
cache.update_conv_state(conv_states, layer_idx=linear_idx)

from transformers.cache_utils import LinearAttentionCacheLayerMixin

if isinstance(cache.layers[layer_idx], LinearAttentionCacheLayerMixin):
    cache.update_conv_state(conv_states, layer_idx=layer_idx)

from transformers.cache_utils import LinearAttentionCacheLayerMixin

def supports_conv_state(cache, layer_idx: int) -> bool:
    return isinstance(cache.layers[layer_idx], LinearAttentionCacheLayerMixin)

try:
    cache.update_conv_state(conv_states, layer_idx=idx)
except ValueError as e:
    if "non-LinearAttention layer" in str(e):
        pass  # expected for attention layers; skip
    else:
        raise

When it happens

Trigger: Calling cache.update_conv_state(conv_states, layer_idx=i) where self.layers[i] is an attention layer (CacheLayerMixin but not LinearAttentionCacheLayerMixin) — e.g. using a uniform layer_idx mapping over a hybrid model where layer indices do not correspond to linear attention layers.

Common situations: Running hybrid models (alternating attention / linear attention) with code that assumes every layer index has conv states; mixing up layer indexing schemes between the cache and the model's layer_types list.

Related errors

• You called `get_seq_length` on layer index {layer_idx}, but
• You called `has_previous_state` on layer index {layer_idx},
• You called `get_mask_sizes` on layer index {layer_idx}, but
• Cannot call `update_indexer` on layer {layer_idx} which is a
• `get_seq_length` can only be called on Attention layers, and

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/e4f6bfefd507cba1. Report an issue: GitHub.