huggingface/transformers · error · ValueError

Passing a tuple of `past_key_values` is not supported anymor

Error message

Passing a tuple of `past_key_values` is not supported anymore. Please use a `Cache` instance.

What it means

Legacy transformers returned/accepted the cache as a tuple of `(key_states, value_states)` per layer. Modern generation requires the object-oriented `Cache` API (e.g. `DynamicCache`) which supports in-place updates and arbitrary cache layouts. If the value passed under `past_key_values` (or `cache_params`) is a tuple, generate raises.

Source

Thrown at src/transformers/generation/utils.py:1955

        """
        Prepares the cache for generation (if applicable), given `generate`'s parameterization. If a cache is
        instantiated, writes it to `model_kwargs`, under the name expected by the model.
        """

        # TODO @raushan, unify cache arg naming for all models
        is_linear_attn_cache = "mamba" in self.__class__.__name__.lower()
        cache_name = "past_key_values" if not is_linear_attn_cache else "cache_params"

        # Quick escape route 1: if the user specifies a cache, we only need to check for conflicting `generate` arguments
        user_defined_cache = model_kwargs.get(cache_name)
        if user_defined_cache is not None:
            if generation_config.cache_implementation is not None:
                raise ValueError(
                    f"Passing both `cache_implementation` (used to initialize certain caches) and `{cache_name}` (a "
                    "Cache object) is unsupported. Please use only one of the two."
                )
            if isinstance(user_defined_cache, tuple):
                raise ValueError(
                    "Passing a tuple of `past_key_values` is not supported anymore. Please use a `Cache` instance."
                )
            return

        # Quick escape route 2: if the user specifies no cache is to be used. (conflicting arguments are handled in
        # `generation_config.validate()`)
        if generation_config.use_cache is False:
            return

        # Quick escape route 3: model that supply it in `prepare_inputs_for_generation` (mamba, zamba, ...)
        if not self._supports_default_dynamic_cache():
            if generation_config.cache_implementation is not None:
                logger.warning_once(
                    "This model does not support `Cache` instances. `cache_implementation` (set to "
                    f"{generation_config.cache_implementation}) will be ignored.",
                )
            return

View on GitHub (pinned to a597f97485)

Solutions

1. Use a `Cache` instance: `from transformers import DynamicCache; out = model.generate(**inputs, past_key_values=DynamicCache())`.
2. Convert an existing tuple: `cache = DynamicCache.from_legacy_cache(legacy_tuple)`.
3. Stop capturing/serializing caches as tuples; persist `DynamicCache` state instead.
4. If a third-party library hands you tuples, wrap its output with `DynamicCache.from_legacy_cache` at the boundary.

Example fix

# before
out = model.generate(**inputs, past_key_values=old_tuple_cache)  # ValueError: tuple not supported

# after
from transformers import DynamicCache
out = model.generate(**inputs, past_key_values=DynamicCache.from_legacy_cache(old_tuple_cache))

from transformers import DynamicCache
if isinstance(past_key_values, tuple):
    past_key_values = DynamicCache.from_legacy_cache(past_key_values)

def is_modern_cache(obj) -> bool:
    return obj is None or not isinstance(obj, tuple)  # Cache instances pass; legacy tuples fail

When it happens

Trigger: `model.generate(**inputs, past_key_values=legacy_tuple)`; passing a manually built `tuple` of per-layer key/value tensors; forwarding cache output captured from an old transformers version or legacy code path.

Common situations: Code written for transformers < 4.36 after an upgrade; caches serialized to disk in tuple form; manual prompt-prefilling code that constructs tuples; copies of old StackOverflow snippets.

Related errors

• You have modified the pretrained model configuration to cont
• Passing both `cache_implementation` (used to initialize cert
• `crop` was called, but the current layer does not track past
• Once the sliding window size has been reached, `DynamicSlidi
• `crop` was called, but the current layer does not track past

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/eeff36103b045946. Report an issue: GitHub.