huggingface/transformers · error · ValueError

`head_dim` was provided as a list of length {len(num_heads)}

Error message

`head_dim` was provided as a list of length {len(num_heads)}, but the Cache currently has {len(self.layers)} layers

What it means

Cache.early_initialization() raises ValueError when head_dim is a list whose length differs from len(self.layers). Like num_heads, per-layer head dims must match the layer count; ints broadcast. Note the f-string reports len(num_heads) rather than len(head_dim) — a formatting slip in the message, the actual check is on head_dim.

Source

Thrown at src/transformers/cache_utils.py:1471

        dtype: torch.dtype,
        device: torch.device,
    ):
        """
        Initialize all the layers in advance (it's otherwise lazily initialized on the first `update` call).
        This is useful for our `export` recipes, as `export` needs everything in advance.
        """
        # To allow different num_heads and head_dim depending on layers, we accept lists
        if isinstance(num_heads, int):
            num_heads = [num_heads] * len(self)
        if isinstance(head_dim, int):
            head_dim = [head_dim] * len(self)

        if len(num_heads) != len(self.layers):
            raise ValueError(
                f"`num_head` was provided as a list of length {len(num_heads)}, but the Cache currently has {len(self.layers)} layers"
            )
        if len(head_dim) != len(self.layers):
            raise ValueError(
                f"`head_dim` was provided as a list of length {len(num_heads)}, but the Cache currently has {len(self.layers)} layers"
            )

        for layer, layer_num_heads, layer_head_dim in zip(self.layers, num_heads, head_dim):
            if not layer.supports_early_init or layer.is_initialized:
                continue
            # Note that the initialization needs all dimensions (except -2), as well as device and dtype, so we use
            # this fake tensor approach. It has size 0 on the -2 dimension, so it does not allocate any data (it only
            # creates an empty tensor with correct shape, dtype and device), which is very efficient and practical
            fake_kv_tensor = torch.zeros((batch_size, layer_num_heads, 0, layer_head_dim), dtype=dtype, device=device)
            # Init the layer
            layer.lazy_initialization(fake_kv_tensor, fake_kv_tensor)

    def get_seq_length(self, layer_idx: int = 0) -> int:
        """Returns the sequence length of the cache for the given layer."""
        if layer_idx >= len(self.layers):
            return 0

View on GitHub (pinned to a597f97485)

Solutions

Make len(head_dim) equal the number of cache layers, or pass a single int
If the message shows a num_heads length you believe is correct, the real mismatch is head_dim — the message interpolates the wrong variable
Derive both lists from the same config used to construct the cache

Example fix

# before
cache.early_initialization(bs, num_heads=8, head_dim=[128, 128], ...)  # cache has 12 layers

# after
cache.early_initialization(bs, num_heads=8, head_dim=128, dtype=torch.bfloat16, device=device)

Defensive patterns

Strategy: validation

Validate before calling

n_layers = len(cache.layers)
if isinstance(head_dim, list):
    assert len(head_dim) == n_layers, f"head_dim list must have {n_layers} entries"
cache.early_initialization(batch_size, num_heads=num_heads, head_dim=head_dim, dtype=dtype, device=device)

Prevention

The error text interpolates len(num_heads) even though head_dim is what mismatched — verify both lists when you see it
Pass a single int head_dim unless the architecture truly varies head_dim per layer

When it happens

Trigger: Calling cache.early_initialization(...) with head_dim=[128, 128] on a cache with e.g. 28 layers. The message will misleadingly show the num_heads length; trust the condition, not the printed variable.

Common situations: Mixed-dimension architectures (varying head_dim per layer) where the head_dim list was built from a stale config; hitting the error after the num_heads check passed and misreading the message as a num_heads problem.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/746af476e091bd40. Report an issue: GitHub.