{"record":{"id":"a32f795f8f1dab49","repo":"huggingface/transformers","slug":"method-is-only-defined-for-dynamic-cache-got","errorCode":null,"errorMessage":"`{method}` is only defined for dynamic cache, got {self.self_attention_cache.__str__()} for the self attention cache and {self.cross_attention_cache.__str__()} for the cross attention cache.","messagePattern":"`(.+?)` is only defined for dynamic cache, got (.+?) for the self attention cache and (.+?) for the cross attention cache\\.","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"src/transformers/cache_utils.py","lineNumber":2045,"sourceCode":"        return self.self_attention_cache.get_max_length(layer_idx)\n\n    def reset(self):\n        self.self_attention_cache.reset()\n        self.cross_attention_cache.reset()\n        for layer_idx in self.is_updated:\n            self.is_updated[layer_idx] = False\n\n    def reorder_cache(self, beam_idx: torch.LongTensor):\n        \"\"\"Reorders the cache for beam search, given the selected beam indices.\"\"\"\n        self.self_attention_cache.reorder_cache(beam_idx)\n        self.cross_attention_cache.reorder_cache(beam_idx)\n\n    def check_dynamic_cache(self, method: str):\n        if not (\n            isinstance(self.self_attention_cache, DynamicCache)\n            and isinstance(self.cross_attention_cache, DynamicCache)\n        ):\n            raise TypeError(\n                f\"`{method}` is only defined for dynamic cache, got {self.self_attention_cache.__str__()} for the self \"\n                f\"attention cache and {self.cross_attention_cache.__str__()} for the cross attention cache.\"\n            )\n\n    @deprecate_kwarg(\"maximum_length\", new_name=\"tokens_to_remove\", version=\"5.18\")\n    def crop(self, tokens_to_remove: int) -> None:\n        \"\"\"\n        Remove `tokens_to_remove` tokens from the current cache layer.\n        \"\"\"\n        self.check_dynamic_cache(self.crop.__name__)\n        self.self_attention_cache.crop(tokens_to_remove)\n\n    def batch_repeat_interleave(self, repeats: int):\n        \"\"\"Repeat the cache `repeats` times in the batch dimension. Used in contrastive search (on the Hub).\"\"\"\n        self.check_dynamic_cache(self.batch_repeat_interleave.__name__)\n        self.self_attention_cache.batch_repeat_interleave(repeats)\n        self.cross_attention_cache.batch_repeat_interleave(repeats)\n","sourceCodeStart":2027,"sourceCodeEnd":2063,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cache_utils.py#L2027-L2063","documentation":"EncoderDecoderCache delegates methods like crop, batch_split, batch_repeat_interleave, and batch_concat to its two inner caches, but these manipulations are only implemented on DynamicCache. check_dynamic_cache() raises TypeError when either self_attention_cache or cross_attention_cache is not a DynamicCache (e.g. StaticCache, QuantizedCache, or OffloadedCache). The message includes the repr of both inner caches so you can see which one is non-dynamic.","triggerScenarios":"Constructing EncoderDecoderCache(StaticCache(...), StaticCache(...)) and then calling .crop(n), .batch_split(...), or .update() paths that require dynamic behavior; calling any method whose first line is self.check_dynamic_cache(...); combining a DynamicCache with a QuantizedCache and calling crop.","commonSituations":"Using StaticCache for performance with an encoder-decoder model (e.g. whisper) and then running beam search / cache trimming that internally calls crop or batch_* helpers; manually assembling an EncoderDecoderCache with custom cache types for offloading or quantization.","solutions":["Use DynamicCache for both slots when you need crop/batch_* operations","Avoid calling the dynamic-only method on a mixed/static setup; construct a fresh cache of the desired length instead of cropping","Upgrade/patch: check the installed transformers version, since newer releases extend these ops to more cache types","If you must crop a StaticCache, rebuild it with a smaller max_length/window instead"],"exampleFix":"// before\nenc = EncoderDecoderCache(StaticCache(config, batch, max_len), StaticCache(config, batch, max_len))\nenc.crop(10)  # TypeError\n\n// after\nfrom transformers import DynamicCache\nenc = EncoderDecoderCache(DynamicCache(), DynamicCache())\n# ... run forward, then crop works\nenc.crop(10)","handlingStrategy":"type-guard","validationCode":"from transformers import DynamicCache\n\ndef can_manipulate(enc_cache) -> bool:\n    return isinstance(enc_cache.self_attention_cache, DynamicCache) and isinstance(\n        enc_cache.cross_attention_cache, DynamicCache\n    )","typeGuard":"from transformers import DynamicCache\n\ndef assert_dynamic(enc_cache, method: str = \"crop\") -> None:\n    if not (\n        isinstance(enc_cache.self_attention_cache, DynamicCache)\n        and isinstance(enc_cache.cross_attention_cache, DynamicCache)\n    ):\n        raise TypeError(f\"{method} requires DynamicCache on both slots\")","tryCatchPattern":"try:\n    enc_cache.crop(n)\nexcept TypeError as e:\n    if \"only defined for dynamic cache\" in str(e):\n        # rebuild a fresh cache instead of cropping a static one\n        enc_cache = make_fresh_cache(max_len - n)\n    else:\n        raise","preventionTips":["Use DynamicCache whenever generation/beam-search cache surgery is expected","Grep your code for .crop(/.batch_split(/.batch_repeat_interleave( on EncoderDecoderCache","Pin a transformers version where the cache types you combine support the ops you need"],"tags":["cache","typeerror","dynamic-cache","static-cache","transformers"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}