huggingface/transformers · error · ValueError

`num_head` was provided as a list of length {len(num_heads)}

Error message

`num_head` was provided as a list of length {len(num_heads)}, but the Cache currently has {len(self.layers)} layers

What it means

Cache.early_initialization() raises ValueError when num_heads is given as a list whose length differs from the number of layers in the cache. Per-layer head counts must align one-to-one with cache layers; ints are broadcast to all layers, lists must match len(self.layers) exactly.

Source

Thrown at src/transformers/cache_utils.py:1467

        self,
        batch_size: int,
        num_heads: int | list[int],
        head_dim: int | list[int],
        dtype: torch.dtype,
        device: torch.device,
    ):
        """
        Initialize all the layers in advance (it's otherwise lazily initialized on the first `update` call).
        This is useful for our `export` recipes, as `export` needs everything in advance.
        """
        # To allow different num_heads and head_dim depending on layers, we accept lists
        if isinstance(num_heads, int):
            num_heads = [num_heads] * len(self)
        if isinstance(head_dim, int):
            head_dim = [head_dim] * len(self)

        if len(num_heads) != len(self.layers):
            raise ValueError(
                f"`num_head` was provided as a list of length {len(num_heads)}, but the Cache currently has {len(self.layers)} layers"
            )
        if len(head_dim) != len(self.layers):
            raise ValueError(
                f"`head_dim` was provided as a list of length {len(num_heads)}, but the Cache currently has {len(self.layers)} layers"
            )

        for layer, layer_num_heads, layer_head_dim in zip(self.layers, num_heads, head_dim):
            if not layer.supports_early_init or layer.is_initialized:
                continue
            # Note that the initialization needs all dimensions (except -2), as well as device and dtype, so we use
            # this fake tensor approach. It has size 0 on the -2 dimension, so it does not allocate any data (it only
            # creates an empty tensor with correct shape, dtype and device), which is very efficient and practical
            fake_kv_tensor = torch.zeros((batch_size, layer_num_heads, 0, layer_head_dim), dtype=dtype, device=device)
            # Init the layer
            layer.lazy_initialization(fake_kv_tensor, fake_kv_tensor)

    def get_seq_length(self, layer_idx: int = 0) -> int:

View on GitHub (pinned to a597f97485)

Solutions

1. Make len(num_heads) equal the number of cache layers, or pass a single int to broadcast
2. Verify against config.get_text_config().num_hidden_layers when using layer_class_to_replicate caches
3. For nested configs, derive num_heads from the same config used to build the cache

Example fix

# before
cache.early_initialization(bs, num_heads=[3, 8], head_dim=[128], ...)  # cache has 28 layers

# after
cache.early_initialization(bs, num_heads=3, head_dim=128, dtype=torch.bfloat16, device=device)

n_layers = len(cache.layers)
if isinstance(num_heads, list):
    assert len(num_heads) == n_layers, f"num_heads list must have {n_layers} entries"
cache.early_initialization(batch_size, num_heads=num_heads, head_dim=head_dim, dtype=dtype, device=device)

When it happens

Trigger: Calling cache.early_initialization(batch_size, num_heads=[3, 8], ...) on a cache whose layer count (from config.num_hidden_layers or the layers list) is not 2. Typical in export flows that pre-initialize everything.

Common situations: Exporting a model with heterogeneous head counts where the num_heads list was built from a different config (text vs. vision sub-config); passing num_heads for one model with a cache built from another; off-by-one when slicing config lists.

Related errors

• `head_dim` was provided as a list of length {len(num_heads)}
• `axis_value` for `HQQ` backend has to be one of [`0`, `1`] b
• You can construct a Cache either from a list `layers` of all
• You should provide exactly one of `layers` or `layer_class_t
• Cannot call `update_conv_state` on a non-LinearAttention lay

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/c8b50ce6020d4d55. Report an issue: GitHub.