{"record":{"id":"af4f26418f1ad4ef","repo":"huggingface/transformers","slug":"crop-was-called-but-the-current-layer-does-not","errorCode":null,"errorMessage":"`crop` was called, but the current layer does not track past states, and the sliding window size was already reached. Call `activate_past_recording` before `crop` to be able to rollback the cache.","messagePattern":"`crop` was called, but the current layer does not track past states, and the sliding window size was already reached\\. Call `activate_past_recording` before `crop` to be able to rollback the cache\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/transformers/cache_utils.py","lineNumber":293,"sourceCode":"    def get_seq_length(self) -> int:\n        \"\"\"Returns the sequence length of the cached states.\"\"\"\n        return self.cumulative_length\n\n    def get_max_length(self) -> int:\n        \"\"\"Return the maximum cache shape of the cache\"\"\"\n        return self.sliding_window\n\n    @deprecate_kwarg(\"max_length\", new_name=\"tokens_to_remove\", version=\"5.18\")\n    def crop(self, tokens_to_remove: int) -> None:\n        \"\"\"\n        Remove `tokens_to_remove` tokens from the current cache layer. This will also restrict the size of the cached states back to their\n        minimal working size, i.e. `sliding_window - 1` if they reached the sliding window length. This means that `crop(0)` will not\n        necessarily always be a no-op, as it may still remove useless states (i.e. states that are not needed for the next `forward`).\n        \"\"\"\n        # If we are beyond the sliding window, we need to be more careful\n        if self.get_seq_length() >= self.sliding_window:\n            if not self.record_past:\n                raise RuntimeError(\n                    \"`crop` was called, but the current layer does not track past states, and the sliding window size was already \"\n                    \"reached. Call `activate_past_recording` before `crop` to be able to rollback the cache.\"\n                )\n            if tokens_to_remove > 0:\n                raise RuntimeError(\n                    \"Once the sliding window size has been reached, `DynamicSlidingWindowLayer` can only be cropped by passing a \"\n                    \"negative int, to specify how many tokens to remove\"\n                )\n            # In this case, simply restrict the size back to sliding window without cropping\n            if tokens_to_remove == 0:\n                self.keys = self.keys[:, :, -self.sliding_window + 1 :, :]\n                self.values = self.values[:, :, -self.sliding_window + 1 :, :]\n            # In this case, we crop and restrict the size back to the sliding window if still larger\n            else:\n                tokens_to_remove = abs(tokens_to_remove)\n                self.keys = self.keys[:, :, -self.sliding_window + 1 - tokens_to_remove : -tokens_to_remove, :]\n                self.values = self.values[:, :, -self.sliding_window + 1 - tokens_to_remove : -tokens_to_remove, :]\n                self.cumulative_length = self.cumulative_length - tokens_to_remove","sourceCodeStart":275,"sourceCodeEnd":311,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cache_utils.py#L275-L311","documentation":"Raised by DynamicSlidingWindowLayer.crop when the cache has already reached the sliding-window length but the layer was created without record_past=True. Beyond the window, the layer keeps only the last sliding_window-1 tokens; rolling back (crop) requires the discarded history, which is only retained when past states are recorded. The error tells you to enable recording before you can crop in that regime.","triggerScenarios":"Creating a DynamicCache with sliding_window set (e.g. for Gemma-2/Mistral sliding attention) without record_past, letting it fill past the window (get_seq_length() >= sliding_window), then calling cache.crop(n) — typically inside assisted decoding / beam search rollback (crop of past states before re-expanding).","commonSituations":"Using generate() with speculative decoding or num_beams>1 on a sliding-window model with a dynamic cache not configured for recording; upgrading transformers where crop(0) is now called to compact the cache after the window fills; manually calling cache.crop on a filled cache.","solutions":["Enable recording: construct the cache with record_past=True (DynamicCache(..., record_past=True) or pass record_past through the cache kwargs) before any forward fills it","Call activate_past_recording() on the cache/layer before cropping, as the message instructs","Only crop while seq_length < sliding_window, where history is still fully retained"],"exampleFix":"# before\ncache = DynamicCache(sliding_window=1024)  # record_past off\n...  # prefill > 1024 tokens\ncache.crop(1)  # RuntimeError\n\n# after\nfrom transformers import DynamicCache\ncache = DynamicCache(sliding_window=1024, record_past=True)\n...  # prefill\ncache.crop(1)  # ok: past states available for rollback","handlingStrategy":"validation","validationCode":"if cache.get_seq_length() >= layer_sliding_window and not layer.record_past:\n    layer.activate_past_recording()  # or construct cache with record_past=True","typeGuard":null,"tryCatchPattern":"try:\n    cache.crop(1)\nexcept RuntimeError as e:\n    if 'activate_past_recording' in str(e):\n        cache.crop(0)  # or re-plan generation without rollback\n    else:\n        raise","preventionTips":["Construct sliding-window caches with record_past=True whenever you use beam search or assisted decoding","Call cache.activate_past_recording() before the first forward if you may need crop()","Keep sequence length below sliding_window if you must crop without recording"],"tags":["cache","kv-cache","sliding-window","generation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}