huggingface/transformers · error · RuntimeError

Linear attention layers can only be cropped by passing a neg

Error message

Linear attention layers can only be cropped by passing a negative int, to specify how many tokens to remove

What it means

LinearAttentionCacheLayerMixin.crop() raises RuntimeError when tokens_to_remove is positive. Unlike attention KV caches where crop removes the last N positive tokens, linear attention layers expect a NEGATIVE int; the sign is a convention marking a linear-attention rollback and the code takes abs(tokens_to_remove) afterwards.

Source

Thrown at src/transformers/cache_utils.py:978

        Calling this function will activate past state recording, meaning that a call to `update_conv_states` will
        wait for a call to `crop` before restricting the size of the `conv_states` to `conv_kernel_size`, to be able
        to retrieve previous full states.
        """
        self.record_past = True

    def crop(self, tokens_to_remove: int):
        """
        Remove `tokens_to_remove` tokens from the current cache layer. This will also restrict the size of the cached states back to their
        minimal working size, i.e. `conv_kernel_size`. This means that `crop(0)` will not necessarily always be a no-op, as it may
        still remove useless states (i.e. states that are not needed for the next `forward`).
        """
        if not self.record_past:
            raise RuntimeError(
                "`crop` was called, but the current layer does not track past states. Call `activate_past_recording` before "
                "`crop` to be able to rollback the cache."
            )
        if tokens_to_remove > 0:
            raise RuntimeError(
                "Linear attention layers can only be cropped by passing a negative int, to specify how many tokens to remove"
            )
        for i in range(self.number_of_states):
            tokens_to_remove = abs(tokens_to_remove)
            # In this case, simply restrict the size back to `conv_kernel_size` without cropping
            if tokens_to_remove == 0:
                self.conv_states[i] = self.conv_states[i][..., -self.conv_kernel_size[i] :]
            # This both crop the last `tokens_to_remove`, as well as resize the conv states to `conv_kernel_size` as we never
            # need more for the next forward
            else:
                self.conv_states[i] = self.conv_states[i][
                    ..., -tokens_to_remove - self.conv_kernel_size[i] : -tokens_to_remove
                ]

    def get_max_length(self) -> int:
        # LinearAttention layer have no sequence length dimension, so simply return -1 here
        return -1

View on GitHub (pinned to a597f97485)

Solutions

1. Pass a negative value: cache.crop(-n) to remove the last n tokens of recorded history
2. Branch on cache type before cropping: use positive semantics for pure attention caches and negative for linear attention layers (Cache.crop dispatches per layer)
3. Check that activate_past_recording() was called, otherwise the sibling RuntimeError at the same site fires first

Example fix

// before
cache.crop(2)

// after
cache.crop(-2)

def safe_crop(cache, n: int):
    for layer in cache.layers:
        if isinstance(layer, LinearAttentionCacheLayerMixin):
            layer.crop(-abs(n))
        else:
            layer.crop(abs(n))

safe_crop(cache, 2)

def is_linear_attention_layer(layer) -> bool:
    return isinstance(layer, LinearAttentionCacheLayerMixin)

try:
    cache.crop(n)
except RuntimeError as e:
    if "negative int" in str(e):
        cache.crop(-abs(n))
    else:
        raise

When it happens

Trigger: Calling crop(positive_int) on a layer of type LinearAttentionCacheLayerMixin, e.g. cache.crop(2) instead of cache.crop(-2). Happens when code written for DynamicCache.crop(n) (which uses positive counts) is reused on a linear-attention cache.

Common situations: Shared helper code that crops caches uniformly across model types during generation; manually trimming cache state between decoding steps; porting DynamicCache examples to hybrid (attention + linear attention) models like Falcon-H1, Qwen3-Next or Zamba.

Related errors

• `crop` was called, but the current layer does not track past
• Cannot call `update_conv_state` on a non-LinearAttention lay
• You called `get_seq_length` on layer index {layer_idx}, but
• `get_seq_length` can only be called on Attention layers, and
• `has_previous_state` can only be called on LinearAttention l

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/e3a6013246c221ef. Report an issue: GitHub.