huggingface/transformers · error · RuntimeError

Once the sliding window size has been reached, `DynamicSlidi

Error message

Once the sliding window size has been reached, `DynamicSlidingWindowLayer` can only be cropped by passing a negative int, to specify how many tokens to remove

What it means

Raised by DynamicSlidingWindowLayer.crop when the sliding window is full and tokens_to_remove is strictly positive. In the pre-window regime crop(n) means 'keep n tokens' (standard DynamicCache semantics); once the window is full that meaning is invalid — removal is specified with a negative count. The full-cache branch only accepts tokens_to_remove <= 0 (0 compacts to the minimal working size, negative values drop that many tokens).

Source

Thrown at src/transformers/cache_utils.py:298

        """Return the maximum cache shape of the cache"""
        return self.sliding_window

    @deprecate_kwarg("max_length", new_name="tokens_to_remove", version="5.18")
    def crop(self, tokens_to_remove: int) -> None:
        """
        Remove `tokens_to_remove` tokens from the current cache layer. This will also restrict the size of the cached states back to their
        minimal working size, i.e. `sliding_window - 1` if they reached the sliding window length. This means that `crop(0)` will not
        necessarily always be a no-op, as it may still remove useless states (i.e. states that are not needed for the next `forward`).
        """
        # If we are beyond the sliding window, we need to be more careful
        if self.get_seq_length() >= self.sliding_window:
            if not self.record_past:
                raise RuntimeError(
                    "`crop` was called, but the current layer does not track past states, and the sliding window size was already "
                    "reached. Call `activate_past_recording` before `crop` to be able to rollback the cache."
                )
            if tokens_to_remove > 0:
                raise RuntimeError(
                    "Once the sliding window size has been reached, `DynamicSlidingWindowLayer` can only be cropped by passing a "
                    "negative int, to specify how many tokens to remove"
                )
            # In this case, simply restrict the size back to sliding window without cropping
            if tokens_to_remove == 0:
                self.keys = self.keys[:, :, -self.sliding_window + 1 :, :]
                self.values = self.values[:, :, -self.sliding_window + 1 :, :]
            # In this case, we crop and restrict the size back to the sliding window if still larger
            else:
                tokens_to_remove = abs(tokens_to_remove)
                self.keys = self.keys[:, :, -self.sliding_window + 1 - tokens_to_remove : -tokens_to_remove, :]
                self.values = self.values[:, :, -self.sliding_window + 1 - tokens_to_remove : -tokens_to_remove, :]
                self.cumulative_length = self.cumulative_length - tokens_to_remove

        # If we did not reach the sliding window, we can do the same as for a full attention layer
        else:
            super().crop(tokens_to_remove)
            self.cumulative_length = self.keys.shape[-2]

View on GitHub (pinned to a597f97485)

Solutions

  1. Once the window is full, pass a negative count: cache.crop(-n) removes n tokens
  2. Use cache.crop(0) to compact back to the minimal working size (sliding_window - 1 tokens) without removing anything
  3. Branch on cache.get_seq_length() >= layer.sliding_window to choose the sign convention

Example fix

# before (window already full)
cache.crop(5)  # positive -> RuntimeError

# after
cache.crop(-5)  # remove 5 tokens
cache.crop(0)   # or just compact to minimal working size
Defensive patterns

Strategy: validation

Validate before calling

if cache.get_seq_length() >= layer_sliding_window:
    to_remove = -num_tokens_to_drop  # negative in the full-window regime
else:
    to_remove = keep_length  # pre-window: crop-to-length semantics
cache.crop(to_remove)

Prevention

When it happens

Trigger: Calling cache.crop(k) with k > 0 after the layer's sequence length has reached sliding_window — e.g. generate() internals or user code applying prefill-style crop semantics (crop(past_length - keep_length)) to an already-full sliding layer.

Common situations: Porting manual cache-management code written for DynamicCache to a sliding-window DynamicCache; beam-search / assisted-decoding helpers that call crop with positive keep-lengths; version upgrades where the crop contract changed to support sliding windows.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/ce3eb4c0cad95bd3. Report an issue: GitHub.