huggingface/transformers · error · TypeError

`{method}` is only defined for dynamic cache, got {self.self

Error message

`{method}` is only defined for dynamic cache, got {self.self_attention_cache.__str__()} for the self attention cache and {self.cross_attention_cache.__str__()} for the cross attention cache.

What it means

EncoderDecoderCache delegates methods like crop, batch_split, batch_repeat_interleave, and batch_concat to its two inner caches, but these manipulations are only implemented on DynamicCache. check_dynamic_cache() raises TypeError when either self_attention_cache or cross_attention_cache is not a DynamicCache (e.g. StaticCache, QuantizedCache, or OffloadedCache). The message includes the repr of both inner caches so you can see which one is non-dynamic.

Source

Thrown at src/transformers/cache_utils.py:2045

        return self.self_attention_cache.get_max_length(layer_idx)

    def reset(self):
        self.self_attention_cache.reset()
        self.cross_attention_cache.reset()
        for layer_idx in self.is_updated:
            self.is_updated[layer_idx] = False

    def reorder_cache(self, beam_idx: torch.LongTensor):
        """Reorders the cache for beam search, given the selected beam indices."""
        self.self_attention_cache.reorder_cache(beam_idx)
        self.cross_attention_cache.reorder_cache(beam_idx)

    def check_dynamic_cache(self, method: str):
        if not (
            isinstance(self.self_attention_cache, DynamicCache)
            and isinstance(self.cross_attention_cache, DynamicCache)
        ):
            raise TypeError(
                f"`{method}` is only defined for dynamic cache, got {self.self_attention_cache.__str__()} for the self "
                f"attention cache and {self.cross_attention_cache.__str__()} for the cross attention cache."
            )

    @deprecate_kwarg("maximum_length", new_name="tokens_to_remove", version="5.18")
    def crop(self, tokens_to_remove: int) -> None:
        """
        Remove `tokens_to_remove` tokens from the current cache layer.
        """
        self.check_dynamic_cache(self.crop.__name__)
        self.self_attention_cache.crop(tokens_to_remove)

    def batch_repeat_interleave(self, repeats: int):
        """Repeat the cache `repeats` times in the batch dimension. Used in contrastive search (on the Hub)."""
        self.check_dynamic_cache(self.batch_repeat_interleave.__name__)
        self.self_attention_cache.batch_repeat_interleave(repeats)
        self.cross_attention_cache.batch_repeat_interleave(repeats)

View on GitHub (pinned to a597f97485)

Solutions

1. Use DynamicCache for both slots when you need crop/batch_* operations
2. Avoid calling the dynamic-only method on a mixed/static setup; construct a fresh cache of the desired length instead of cropping
3. Upgrade/patch: check the installed transformers version, since newer releases extend these ops to more cache types
4. If you must crop a StaticCache, rebuild it with a smaller max_length/window instead

Example fix

// before
enc = EncoderDecoderCache(StaticCache(config, batch, max_len), StaticCache(config, batch, max_len))
enc.crop(10)  # TypeError

// after
from transformers import DynamicCache
enc = EncoderDecoderCache(DynamicCache(), DynamicCache())
# ... run forward, then crop works
enc.crop(10)

from transformers import DynamicCache

def can_manipulate(enc_cache) -> bool:
    return isinstance(enc_cache.self_attention_cache, DynamicCache) and isinstance(
        enc_cache.cross_attention_cache, DynamicCache
    )

from transformers import DynamicCache

def assert_dynamic(enc_cache, method: str = "crop") -> None:
    if not (
        isinstance(enc_cache.self_attention_cache, DynamicCache)
        and isinstance(enc_cache.cross_attention_cache, DynamicCache)
    ):
        raise TypeError(f"{method} requires DynamicCache on both slots")

try:
    enc_cache.crop(n)
except TypeError as e:
    if "only defined for dynamic cache" in str(e):
        # rebuild a fresh cache instead of cropping a static one
        enc_cache = make_fresh_cache(max_len - n)
    else:
        raise

When it happens

Trigger: Constructing EncoderDecoderCache(StaticCache(...), StaticCache(...)) and then calling .crop(n), .batch_split(...), or .update() paths that require dynamic behavior; calling any method whose first line is self.check_dynamic_cache(...); combining a DynamicCache with a QuantizedCache and calling crop.

Common situations: Using StaticCache for performance with an encoder-decoder model (e.g. whisper) and then running beam search / cache trimming that internally calls crop or batch_* helpers; manually assembling an EncoderDecoderCache with custom cache types for offloading or quantization.

Related errors

• One of the two arguments is not a Cache: {type(caches[0]) =
• Expected 1 or 2 arguments, got {len(caches)}
• `crop` was called, but the current layer does not track past
• Once the sliding window size has been reached, `DynamicSlidi
• `axis_value` for `HQQ` backend has to be one of [`0`, `1`] b

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/a32f795f8f1dab49. Report an issue: GitHub.