huggingface/transformers · error · RuntimeError

`crop` was called, but the current layer does not track past

Error message

`crop` was called, but the current layer does not track past states. Call `activate_past_recording` before `crop` to be able to rollback the cache.

What it means

LinearAttentionCacheLayerMixin.crop() raises RuntimeError when crop() is called while record_past is False. Linear attention layers keep fixed-size conv/recurrent states that are trimmed to conv_kernel_size after each forward; to support rollback (beam search, assisted decoding) the layer must retain extra history, which is only enabled by calling activate_past_recording() first.

Source

Thrown at src/transformers/cache_utils.py:973

            if self.is_recurrent_states_initialized[i]:
                self.recurrent_states[i] = self.recurrent_states[i].index_select(0, beam_idx.to(self.device))

    def activate_past_recording(self):
        """
        Calling this function will activate past state recording, meaning that a call to `update_conv_states` will
        wait for a call to `crop` before restricting the size of the `conv_states` to `conv_kernel_size`, to be able
        to retrieve previous full states.
        """
        self.record_past = True

    def crop(self, tokens_to_remove: int):
        """
        Remove `tokens_to_remove` tokens from the current cache layer. This will also restrict the size of the cached states back to their
        minimal working size, i.e. `conv_kernel_size`. This means that `crop(0)` will not necessarily always be a no-op, as it may
        still remove useless states (i.e. states that are not needed for the next `forward`).
        """
        if not self.record_past:
            raise RuntimeError(
                "`crop` was called, but the current layer does not track past states. Call `activate_past_recording` before "
                "`crop` to be able to rollback the cache."
            )
        if tokens_to_remove > 0:
            raise RuntimeError(
                "Linear attention layers can only be cropped by passing a negative int, to specify how many tokens to remove"
            )
        for i in range(self.number_of_states):
            tokens_to_remove = abs(tokens_to_remove)
            # In this case, simply restrict the size back to `conv_kernel_size` without cropping
            if tokens_to_remove == 0:
                self.conv_states[i] = self.conv_states[i][..., -self.conv_kernel_size[i] :]
            # This both crop the last `tokens_to_remove`, as well as resize the conv states to `conv_kernel_size` as we never
            # need more for the next forward
            else:
                self.conv_states[i] = self.conv_states[i][
                    ..., -tokens_to_remove - self.conv_kernel_size[i] : -tokens_to_remove
                ]

View on GitHub (pinned to a597f97485)

Solutions

1. Call cache.activate_past_recording() before the forward pass that precedes crop (it sets record_past=True on all layers that support it)
2. If using generate(), prefer a decoding strategy that does not need rollback (e.g. greedy/multinomial sampling) for linear-attention models
3. Do not call crop() on a fresh cache that has never run a forward — recording is only meaningful after states exist

Example fix

// before
out = model(**inputs, cache=cache)
cache.crop(-2)  # RuntimeError

// after
cache.activate_past_recording()
out = model(**inputs, cache=cache)
cache.crop(-2)

if any(
    isinstance(l, LinearAttentionCacheLayerMixin) and not l.record_past
    for l in cache.layers
):
    cache.activate_past_recording()
out = model(**inputs, cache=cache)
cache.crop(-2)

try:
    cache.crop(-n)
except RuntimeError as e:
    if "activate_past_recording" in str(e):
        cache.activate_past_recording()  # re-record from next forward, then retry after a step
    else:
        raise

When it happens

Trigger: Calling cache.crop(...) (e.g. via Cache.crop or beam-search rollback paths in generation) on a cache whose linear attention layers never had activate_past_recording() called. Generation code paths that call crop on caches (beam search, and _supports_default_streaming_layout rollback) hit this when the cache was created fresh without recording enabled.

Common situations: Using generate() with beam search or any strategy that rewinds the cache on a model with linear attention / conv states (e.g. Mamba-style or hybrid models) without enabling past recording; manually calling DynamicCache-style crop APIs on a linear-attention cache.

Related errors

• Linear attention layers can only be cropped by passing a neg
• `crop` was called, but the current layer does not track past
• Once the sliding window size has been reached, `DynamicSlidi
• Cannot call `update_conv_state` on a non-LinearAttention lay
• You called `get_seq_length` on layer index {layer_idx}, but

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/bd58762eb0c745c9. Report an issue: GitHub.