huggingface/transformers · error · RuntimeError

`crop` was called, but the current layer does not track past

Error message

`crop` was called, but the current layer does not track past states, and the sliding window size was already reached. Call `activate_past_recording` before `crop` to be able to rollback the cache.

What it means

Raised by DynamicSlidingWindowLayer.crop when the cache has already reached the sliding-window length but the layer was created without record_past=True. Beyond the window, the layer keeps only the last sliding_window-1 tokens; rolling back (crop) requires the discarded history, which is only retained when past states are recorded. The error tells you to enable recording before you can crop in that regime.

Source

Thrown at src/transformers/cache_utils.py:293

    def get_seq_length(self) -> int:
        """Returns the sequence length of the cached states."""
        return self.cumulative_length

    def get_max_length(self) -> int:
        """Return the maximum cache shape of the cache"""
        return self.sliding_window

    @deprecate_kwarg("max_length", new_name="tokens_to_remove", version="5.18")
    def crop(self, tokens_to_remove: int) -> None:
        """
        Remove `tokens_to_remove` tokens from the current cache layer. This will also restrict the size of the cached states back to their
        minimal working size, i.e. `sliding_window - 1` if they reached the sliding window length. This means that `crop(0)` will not
        necessarily always be a no-op, as it may still remove useless states (i.e. states that are not needed for the next `forward`).
        """
        # If we are beyond the sliding window, we need to be more careful
        if self.get_seq_length() >= self.sliding_window:
            if not self.record_past:
                raise RuntimeError(
                    "`crop` was called, but the current layer does not track past states, and the sliding window size was already "
                    "reached. Call `activate_past_recording` before `crop` to be able to rollback the cache."
                )
            if tokens_to_remove > 0:
                raise RuntimeError(
                    "Once the sliding window size has been reached, `DynamicSlidingWindowLayer` can only be cropped by passing a "
                    "negative int, to specify how many tokens to remove"
                )
            # In this case, simply restrict the size back to sliding window without cropping
            if tokens_to_remove == 0:
                self.keys = self.keys[:, :, -self.sliding_window + 1 :, :]
                self.values = self.values[:, :, -self.sliding_window + 1 :, :]
            # In this case, we crop and restrict the size back to the sliding window if still larger
            else:
                tokens_to_remove = abs(tokens_to_remove)
                self.keys = self.keys[:, :, -self.sliding_window + 1 - tokens_to_remove : -tokens_to_remove, :]
                self.values = self.values[:, :, -self.sliding_window + 1 - tokens_to_remove : -tokens_to_remove, :]
                self.cumulative_length = self.cumulative_length - tokens_to_remove

View on GitHub (pinned to a597f97485)

Solutions

Enable recording: construct the cache with record_past=True (DynamicCache(..., record_past=True) or pass record_past through the cache kwargs) before any forward fills it
Call activate_past_recording() on the cache/layer before cropping, as the message instructs
Only crop while seq_length < sliding_window, where history is still fully retained

Example fix

# before
cache = DynamicCache(sliding_window=1024)  # record_past off
...  # prefill > 1024 tokens
cache.crop(1)  # RuntimeError

# after
from transformers import DynamicCache
cache = DynamicCache(sliding_window=1024, record_past=True)
...  # prefill
cache.crop(1)  # ok: past states available for rollback

Defensive patterns

Strategy: validation

Validate before calling

if cache.get_seq_length() >= layer_sliding_window and not layer.record_past:
    layer.activate_past_recording()  # or construct cache with record_past=True

Try / catch

try:
    cache.crop(1)
except RuntimeError as e:
    if 'activate_past_recording' in str(e):
        cache.crop(0)  # or re-plan generation without rollback
    else:
        raise

Prevention

Construct sliding-window caches with record_past=True whenever you use beam search or assisted decoding
Call cache.activate_past_recording() before the first forward if you may need crop()
Keep sequence length below sliding_window if you must crop without recording

When it happens

Trigger: Creating a DynamicCache with sliding_window set (e.g. for Gemma-2/Mistral sliding attention) without record_past, letting it fill past the window (get_seq_length() >= sliding_window), then calling cache.crop(n) — typically inside assisted decoding / beam search rollback (crop of past states before re-expanding).

Common situations: Using generate() with speculative decoding or num_beams>1 on a sliding-window model with a dynamic cache not configured for recording; upgrading transformers where crop(0) is now called to compact the cache after the window fills; manually calling cache.crop on a filled cache.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/af4f26418f1ad4ef. Report an issue: GitHub.