huggingface/transformers · error · AssertionError

The model must have a generation config to be exported with

Error message

The model must have a generation config to be exported with static caching. Please set `generation_config` in `model`.

What it means

Error "The model must have a generation config to be exported with static caching. Please set `generation_config` in `model`." thrown in huggingface/transformers.

Source

Thrown at src/transformers/integrations/executorch.py:510

                in `generation_config.cache_config` and otherwise we raise a ValueError.
            max_cache_len (`Optional[int]`): The maximum cache length for generation. Same mechanism as `batch_size` if
                not provided.
            device (`Optional[torch.device]`): The device to use. If not provided, we check if a value can be found
                in `generation_config.cache_config` and otherwise we use `model.device` (no error is raised).

        Raises:
            AssertionError: If the pretrained model does not have caching enabled or if it does
            not use a 'static' caching implementation in `model.generation_config`.
            ValueError: If `batch_size` or `max_cache_len` is not provided, either as an argument or in `cache_config`.
        """
        super().__init__()

        config = model.config.get_text_config()
        generation_config = model.generation_config

        # Sanity checks
        if generation_config is None:
            raise AssertionError(
                "The model must have a generation config to be exported with static caching. "
                "Please set `generation_config` in `model`."
            )
        if not generation_config.use_cache:
            raise AssertionError(
                "The model must have caching enabled to be exported with static caching. "
                "Please set `generation_config.use_cache=True`."
            )
        if generation_config.cache_implementation != "static":
            raise AssertionError(
                "The model must use a 'static' caching implementation to be exported with static caching. "
                "Please set `generation_config.cache_implementation='static'`."
            )

        cache_config = {} if generation_config.cache_config is None else generation_config.cache_config

        # Ensure batch_size and max_cache_len are set
        if batch_size is None:

View on GitHub (pinned to a597f97485)

Solutions

1. Set `model.generation_config` before exporting with static caching.

When it happens

Trigger: Raised in ExecuTorch static-cache export when the model has no generation_config.

Common situations: Exporting a bare model to ExecuTorch static cache without attaching a generation_config.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/33cf13c482c7b8ae. Report an issue: GitHub.