{"record":{"id":"ce3eb4c0cad95bd3","repo":"huggingface/transformers","slug":"once-the-sliding-window-size-has-been-reached-dy","errorCode":null,"errorMessage":"Once the sliding window size has been reached, `DynamicSlidingWindowLayer` can only be cropped by passing a negative int, to specify how many tokens to remove","messagePattern":"Once the sliding window size has been reached, `DynamicSlidingWindowLayer` can only be cropped by passing a negative int, to specify how many tokens to remove","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/transformers/cache_utils.py","lineNumber":298,"sourceCode":"        \"\"\"Return the maximum cache shape of the cache\"\"\"\n        return self.sliding_window\n\n    @deprecate_kwarg(\"max_length\", new_name=\"tokens_to_remove\", version=\"5.18\")\n    def crop(self, tokens_to_remove: int) -> None:\n        \"\"\"\n        Remove `tokens_to_remove` tokens from the current cache layer. This will also restrict the size of the cached states back to their\n        minimal working size, i.e. `sliding_window - 1` if they reached the sliding window length. This means that `crop(0)` will not\n        necessarily always be a no-op, as it may still remove useless states (i.e. states that are not needed for the next `forward`).\n        \"\"\"\n        # If we are beyond the sliding window, we need to be more careful\n        if self.get_seq_length() >= self.sliding_window:\n            if not self.record_past:\n                raise RuntimeError(\n                    \"`crop` was called, but the current layer does not track past states, and the sliding window size was already \"\n                    \"reached. Call `activate_past_recording` before `crop` to be able to rollback the cache.\"\n                )\n            if tokens_to_remove > 0:\n                raise RuntimeError(\n                    \"Once the sliding window size has been reached, `DynamicSlidingWindowLayer` can only be cropped by passing a \"\n                    \"negative int, to specify how many tokens to remove\"\n                )\n            # In this case, simply restrict the size back to sliding window without cropping\n            if tokens_to_remove == 0:\n                self.keys = self.keys[:, :, -self.sliding_window + 1 :, :]\n                self.values = self.values[:, :, -self.sliding_window + 1 :, :]\n            # In this case, we crop and restrict the size back to the sliding window if still larger\n            else:\n                tokens_to_remove = abs(tokens_to_remove)\n                self.keys = self.keys[:, :, -self.sliding_window + 1 - tokens_to_remove : -tokens_to_remove, :]\n                self.values = self.values[:, :, -self.sliding_window + 1 - tokens_to_remove : -tokens_to_remove, :]\n                self.cumulative_length = self.cumulative_length - tokens_to_remove\n\n        # If we did not reach the sliding window, we can do the same as for a full attention layer\n        else:\n            super().crop(tokens_to_remove)\n            self.cumulative_length = self.keys.shape[-2]","sourceCodeStart":280,"sourceCodeEnd":316,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cache_utils.py#L280-L316","documentation":"Raised by DynamicSlidingWindowLayer.crop when the sliding window is full and tokens_to_remove is strictly positive. In the pre-window regime crop(n) means 'keep n tokens' (standard DynamicCache semantics); once the window is full that meaning is invalid — removal is specified with a negative count. The full-cache branch only accepts tokens_to_remove <= 0 (0 compacts to the minimal working size, negative values drop that many tokens).","triggerScenarios":"Calling cache.crop(k) with k > 0 after the layer's sequence length has reached sliding_window — e.g. generate() internals or user code applying prefill-style crop semantics (crop(past_length - keep_length)) to an already-full sliding layer.","commonSituations":"Porting manual cache-management code written for DynamicCache to a sliding-window DynamicCache; beam-search / assisted-decoding helpers that call crop with positive keep-lengths; version upgrades where the crop contract changed to support sliding windows.","solutions":["Once the window is full, pass a negative count: cache.crop(-n) removes n tokens","Use cache.crop(0) to compact back to the minimal working size (sliding_window - 1 tokens) without removing anything","Branch on cache.get_seq_length() >= layer.sliding_window to choose the sign convention"],"exampleFix":"# before (window already full)\ncache.crop(5)  # positive -> RuntimeError\n\n# after\ncache.crop(-5)  # remove 5 tokens\ncache.crop(0)   # or just compact to minimal working size","handlingStrategy":"validation","validationCode":"if cache.get_seq_length() >= layer_sliding_window:\n    to_remove = -num_tokens_to_drop  # negative in the full-window regime\nelse:\n    to_remove = keep_length  # pre-window: crop-to-length semantics\ncache.crop(to_remove)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Learn the two crop regimes: pre-window crop(n) keeps n tokens; full-window crop(-n) drops n","Branch on get_seq_length() vs sliding_window before calling crop","Prefer cache.crop(0) for compaction after the window fills"],"tags":["cache","kv-cache","sliding-window","generation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}