huggingface/transformers · error · AssertionError

Model must have caching enabled.

Error message

Model must have caching enabled.

What it means

Error "Model must have caching enabled." thrown in huggingface/transformers.

Source

Thrown at src/transformers/integrations/executorch.py:703

            device (`Optional[torch.device]`): The device to use. If not provided, we check if a value can be found
                in `generation_config.cache_config` and otherwise we use `model.device` (no error is raised).
        Raises:
            AssertionError: If the model doesn't have the expected configuration for hybrid StaticCache.
            ValueError: If `batch_size` or `max_cache_len` is not provided, either as an argument or in `cache_config`.
        """
        super().__init__()
        self.model = model
        config = model.config.get_text_config()
        generation_config = model.generation_config

        # Sanity checks
        if generation_config is None:
            raise AssertionError(
                "The model must have a generation config to be exported with static caching. "
                "Please set `generation_config` in `model`."
            )
        if not config.use_cache:
            raise AssertionError("Model must have caching enabled.")

        cache_config = {} if generation_config.cache_config is None else generation_config.cache_config
        # Ensure batch_size and max_cache_len are set
        if batch_size is None:
            batch_size = cache_config.get("batch_size", None)
            if batch_size is None:
                raise ValueError("batch_size must be provided, either as an argument or in cache_config.")
        if max_cache_len is None:
            max_cache_len = cache_config.get("max_cache_len", None)
            if max_cache_len is None:
                raise ValueError("max_cache_len must be provided, either as an argument or in cache_config.")
        # Infer device if not provided
        if device is None:
            device = cache_config.get("device", model.device)

        # Initialize the cache
        self.cache = StaticCache(config=config, max_cache_len=max_cache_len)
        # Since StaticSlidingWindow have dynamic control flow that cannot be avoided, we have to replace them here by

View on GitHub (pinned to a597f97485)

Solutions

1. Enable caching on the model (`use_cache=True`) before export.

When it happens

Trigger: Raised in ExecuTorch integration when the model's caching is disabled.

Common situations: Running ExecuTorch export/inference on a model configured with use_cache=False.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/c6869fc055f8870e. Report an issue: GitHub.