huggingface/transformers · error · ValueError

Passing both `cache_implementation` (used to initialize cert

Error message

Passing both `cache_implementation` (used to initialize certain caches) and `{cache_name}` (a Cache object) is unsupported. Please use only one of the two.

What it means

A generation cache can come either from `generation_config.cache_implementation` (a string like 'dynamic'/'static'/'quantized' that makes generate BUILD a cache) or from a user-supplied `Cache` object in `model_kwargs` (`past_key_values`, or `cache_params` for mamba-style models). Supplying both is ambiguous, so when `model_kwargs[cache_name]` is set and `cache_implementation` is not None, generate raises.

Source

Thrown at src/transformers/generation/utils.py:1950

        model_kwargs: dict,
        generation_mode: GenerationMode,
        batch_size: int,
        max_cache_length: int,
    ) -> bool:
        """
        Prepares the cache for generation (if applicable), given `generate`'s parameterization. If a cache is
        instantiated, writes it to `model_kwargs`, under the name expected by the model.
        """

        # TODO @raushan, unify cache arg naming for all models
        is_linear_attn_cache = "mamba" in self.__class__.__name__.lower()
        cache_name = "past_key_values" if not is_linear_attn_cache else "cache_params"

        # Quick escape route 1: if the user specifies a cache, we only need to check for conflicting `generate` arguments
        user_defined_cache = model_kwargs.get(cache_name)
        if user_defined_cache is not None:
            if generation_config.cache_implementation is not None:
                raise ValueError(
                    f"Passing both `cache_implementation` (used to initialize certain caches) and `{cache_name}` (a "
                    "Cache object) is unsupported. Please use only one of the two."
                )
            if isinstance(user_defined_cache, tuple):
                raise ValueError(
                    "Passing a tuple of `past_key_values` is not supported anymore. Please use a `Cache` instance."
                )
            return

        # Quick escape route 2: if the user specifies no cache is to be used. (conflicting arguments are handled in
        # `generation_config.validate()`)
        if generation_config.use_cache is False:
            return

        # Quick escape route 3: model that supply it in `prepare_inputs_for_generation` (mamba, zamba, ...)
        if not self._supports_default_dynamic_cache():
            if generation_config.cache_implementation is not None:
                logger.warning_once(

View on GitHub (pinned to a597f97485)

Solutions

1. Pick one mechanism: either pass the `Cache` object and clear `cache_implementation` (`model.generate(..., past_key_values=cache, cache_implementation=None)`),
2. or drop the Cache object and keep `cache_implementation` so generate constructs the cache.
3. Check `model.generation_config.cache_implementation` — it may be set from a saved generation_config.json even if you never set it in the call.
4. Note mamba/linear-attention models use the kwarg name `cache_params`, not `past_key_values`.

Example fix

# before
cache = DynamicCache()
out = model.generate(**inputs, past_key_values=cache, cache_implementation="dynamic")  # ValueError: both

# after
out = model.generate(**inputs, past_key_values=cache, cache_implementation=None)

cache_name = "cache_params" if "mamba" in type(model).__name__.lower() else "past_key_values"
if kwargs.get(cache_name) is not None and (kwargs.get("cache_implementation") or model.generation_config.cache_implementation):
    kwargs["cache_implementation"] = None  # or drop kwargs[cache_name]

When it happens

Trigger: `model.generate(**inputs, past_key_values=DynamicCache(), cache_implementation="dynamic")`, or `model.generate(..., past_key_values=cache)` while `model.generation_config.cache_implementation = "static"` was set earlier.

Common situations: Serving code that pre-creates caches for throughput combined with a generation_config.json or wrapper that sets `cache_implementation`; incremental upgrades adding `cache_implementation` globally while callers still pass caches; mamba-family models where the key is `cache_params`.

Related errors

• Passing a tuple of `past_key_values` is not supported anymor
• `crop` was called, but the current layer does not track past
• Once the sliding window size has been reached, `DynamicSlidi
• `crop` was called, but the current layer does not track past
• This model does not support the quantized cache. If you want

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/082f2a3e1bda1035. Report an issue: GitHub.