huggingface/transformers · error · ValueError

max_cache_len must be provided, either as an argument or in

Error message

max_cache_len must be provided, either as an argument or in cache_config.

What it means

Error "max_cache_len must be provided, either as an argument or in cache_config." thrown in huggingface/transformers.

Source

Thrown at src/transformers/integrations/executorch.py:535

                "Please set `generation_config.use_cache=True`."
            )
        if generation_config.cache_implementation != "static":
            raise AssertionError(
                "The model must use a 'static' caching implementation to be exported with static caching. "
                "Please set `generation_config.cache_implementation='static'`."
            )

        cache_config = {} if generation_config.cache_config is None else generation_config.cache_config

        # Ensure batch_size and max_cache_len are set
        if batch_size is None:
            batch_size = cache_config.get("batch_size", None)
            if batch_size is None:
                raise ValueError("batch_size must be provided, either as an argument or in cache_config.")
        if max_cache_len is None:
            max_cache_len = cache_config.get("max_cache_len", None)
            if max_cache_len is None:
                raise ValueError("max_cache_len must be provided, either as an argument or in cache_config.")
        # Infer device if not provided
        if device is None:
            device = cache_config.get("device", model.device)

        # Initialize the static cache
        self.model = model
        self.static_cache = StaticCache(max_cache_len=max_cache_len, config=config)
        # Since StaticSlidingWindow have dynamic control flow that cannot be avoided, we have to replace them here by
        # simple StaticLayer... It means that any generation beyond the window is unfortunately unsupported
        for i, layer in enumerate(self.static_cache.layers):
            if isinstance(layer, StaticSlidingWindowLayer):
                self.static_cache.layers[i] = StaticLayer(max_cache_len)
        num_heads, head_dim = get_head_shapes(config)
        dtype = self.model.dtype
        # We need this call to initialize all the layers (otherwise it's done lazily, which is not exportable)
        self.static_cache.early_initialization(batch_size, num_heads, head_dim, dtype, device)

        # Register cache buffers to make them exportable

View on GitHub (pinned to a597f97485)

Solutions

1. Pass `max_cache_len` explicitly or include it in `cache_config`.

When it happens

Trigger: Raised in ExecuTorch export when max_cache_len is missing from both the arguments and cache_config.

Common situations: Static export requires a fixed cache length; none was supplied in either location.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/7235abe690180d0b. Report an issue: GitHub.