huggingface/transformers · error · ValueError

Invalid `cache_implementation` ({}). Choose one of: {}

Error message

Invalid `cache_implementation` ({}). Choose one of: {}

What it means

validate() checks cache_implementation against the known cache backends (ALL_CACHE_IMPLEMENTATIONS plus 'paged', e.g. 'static', 'sliding_window', 'mamba', 'paged'). Any other string is rejected because generate() would later fail to construct the cache.

Source

Thrown at src/transformers/generation/configuration_utils.py:687

        if self.max_new_tokens is not None and self.max_new_tokens <= 0:
            raise ValueError(f"`max_new_tokens` must be greater than 0, but is {self.max_new_tokens}.")
        if self.assistant_ensemble_weight is not None and not (0.0 < self.assistant_ensemble_weight < 1.0):
            raise ValueError(
                f"`assistant_ensemble_weight` must be in the open interval `(0.0, 1.0)`, "
                f"but is {self.assistant_ensemble_weight}. Use `None` for standard (lossless) speculative decoding."
            )
        if self.pad_token_id is not None and self.pad_token_id < 0:
            minor_issues["pad_token_id"] = (
                f"`pad_token_id` should be positive but got {self.pad_token_id}. This will cause errors when batch "
                "generating, if there is padding. Please set `pad_token_id` explicitly as "
                "`model.generation_config.pad_token_id=PAD_TOKEN_ID` to avoid errors in generation"
            )
        # 1.2. Cache attributes
        # "paged" re-routes to continuous batching and so it is a valid cache implementation. But we do not want to test
        # it with the `generate` as the other would be, so we we cannot add it to ALL_CACHE_IMPLEMENTATIONS
        valid_cache_implementations = ALL_CACHE_IMPLEMENTATIONS + ("paged",)
        if self.cache_implementation is not None and self.cache_implementation not in valid_cache_implementations:
            raise ValueError(
                f"Invalid `cache_implementation` ({self.cache_implementation}). Choose one of: "
                f"{valid_cache_implementations}"
            )
        if self.max_cache_len is not None and self.cache_implementation not in ALL_STATIC_CACHE_IMPLEMENTATIONS:
            logger.warning_once(
                f"`max_cache_len` is only used with static caches ({STATIC_CACHE_IMPLEMENTATIONS}); it will be "
                f"ignored with `cache_implementation={self.cache_implementation!r}`."
            )
        # 1.3. Performance attributes
        if self.compile_config is not None and not isinstance(self.compile_config, CompileConfig):
            raise ValueError(
                f"You provided `compile_config` as an instance of {type(self.compile_config)}, but it must be an "
                "instance of `CompileConfig`."
            )
        # 1.4. Watermarking attributes
        if self.watermarking_config is not None:
            self.watermarking_config.validate()

View on GitHub (pinned to a597f97485)

Solutions

1. Use one of the names printed in the error message (valid_cache_implementations list)
2. Check your transformers version's ALL_CACHE_IMPLEMENTATIONS for supported names
3. For typo-prone external configs, validate against the list before constructing GenerationConfig

Example fix

# before
cfg = GenerationConfig(cache_implementation="dynamic")
# after (recent versions)
cfg = GenerationConfig(cache_implementation=None)  # default dynamic cache, or "static" for static cache

from transformers.generation.configuration_utils import ALL_CACHE_IMPLEMENTATIONS

def valid_cache_implementation(name) -> bool:
    return name is None or name in set(ALL_CACHE_IMPLEMENTATIONS) | {"paged"}

def is_valid_cache_name(name: str) -> bool:
    from transformers.generation.configuration_utils import ALL_CACHE_IMPLEMENTATIONS
    return name in set(ALL_CACHE_IMPLEMENTATIONS) | {"paged"}

When it happens

Trigger: GenerationConfig(cache_implementation='flash') (typo), 'dynamic' on a version where it was renamed/removed, or a custom cache name not registered in the implementation list.

Common situations: Typos; version drift where cache names changed between transformers releases; copying cache_implementation from tutorials targeting a different version.

Related errors

• `early_stopping` must be a boolean or 'never', but is {}.
• `max_new_tokens` must be greater than 0, but is {}.
• Invalid max_matching_ngram_size or num_output_tokens
• `assistant_ensemble_weight` must be in the open interval `(0
• You provided `compile_config` as an instance of {}, but it m

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/3c116ec150328891. Report an issue: GitHub.