huggingface/transformers · error · ValueError

Expected {len(combined_cache_data) = } to be 4 or 6. {combin

Error message

Expected {len(combined_cache_data) = } to be 4 or 6.
{combined_cache_data = }

What it means

EncoderDecoderCache.__init__ raises ValueError in its single-argument (DDP-style) path when an element of the iterable has length other than 4 or 6. Each per-layer tuple must carry either 6 tensors (self-attn k/v/sliding + cross-attn k/v/sliding) or 4 tensors (legacy: self-attn k/v + cross-attn k/v) so it can be split into self- and cross-attention DynamicCaches.

Source

Thrown at src/transformers/cache_utils.py:1986

    >>> outputs.past_key_values # access cache filled with key/values from generation
    EncoderDecoderCache()
    ```
    """

    def __init__(self, *caches) -> None:
        # For dp and ddp support, if only one argument is passed, it should be an iterable of DynamicCache ddp data
        if len(caches) == 1:
            self_attention_cache_data, cross_attention_cache_data = [], []
            for combined_cache_data in caches[0]:
                if len(combined_cache_data) == 6:  # two tuple of style (self_attn_k, self_attn_v, self_attn_sliding)
                    self_attention_cache_data.append(combined_cache_data[:3])
                    cross_attention_cache_data.append(combined_cache_data[3:])
                # To support old DDP-style init, we handle the case where the tuple has no sliding window tensor
                elif len(combined_cache_data) == 4:  # two tuple of style (self_attn_k, self_attn_v)
                    self_attention_cache_data.append(combined_cache_data[:2])
                    cross_attention_cache_data.append(combined_cache_data[2:])
                else:
                    raise ValueError(f"Expected {len(combined_cache_data) = } to be 4 or 6.\n{combined_cache_data = }")
            self.self_attention_cache = DynamicCache(self_attention_cache_data)
            self.cross_attention_cache = DynamicCache(cross_attention_cache_data)
        # Otherwise, we should get two arguments, a self-attention cache and a cross-attention cache
        elif len(caches) == 2:
            if not isinstance(caches[0], Cache) or not isinstance(caches[1], Cache):
                raise TypeError(f"One of the two arguments is not a Cache: {type(caches[0]) = }, {type(caches[1]) = }")
            self.self_attention_cache = caches[0]
            self.cross_attention_cache = caches[1]
        # Error case
        else:
            raise ValueError(f"Expected 1 or 2 arguments, got {len(caches)}")

        self.is_updated = {}
        for layer_idx in range(len(self.cross_attention_cache)):
            self.is_updated[layer_idx] = bool(self.cross_attention_cache.get_seq_length(layer_idx) > 0)

    def __iter__(self):
        """Returns tuples of style (self_attn_k, self_attn_v, self_attn_sliding, cross_attn_k, cross_attn_v, cross_attn_sliding)"""

View on GitHub (pinned to a597f97485)

Solutions

1. Ensure each per-layer entry is exactly (k_self, v_self, sliding_self, k_cross, v_cross, sliding_cross) or legacy (k_self, v_self, k_cross, v_cross)
2. If your data only has self-attention halves, pad the cross-attention half with None or empty tensors so each tuple is length 4/6
3. Prefer the two-argument form EncoderDecoderCache(self_attention_cache, cross_attention_cache) with real Cache objects instead of raw tuples

Example fix

# before
cache = EncoderDecoderCache([(k, v) for k, v in saved_layers])  # 2-tuples -> ValueError

# after
cache = EncoderDecoderCache(
    DynamicCache([DynamicCacheLayer.from_tensors(k, v) for k, v, _, _, _, _ in saved]),
    DynamicCache([DynamicCacheLayer.from_tensors(k, v) for _, _, _, k, v, _ in saved]),
)

for entry in caches_iterable:
    if not isinstance(entry, tuple) or len(entry) not in (4, 6):
        raise ValueError(f"bad per-layer cache entry: {entry!r}")
cache = EncoderDecoderCache(caches_iterable)

def is_valid_ddp_cache_entry(entry) -> bool:
    return isinstance(entry, (tuple, list)) and len(entry) in (4, 6) and all(hasattr(t, "shape") for t in entry)

try:
    cache = EncoderDecoderCache(caches_iterable)
except ValueError as e:
    if "to be 4 or 6" in str(e):
        # normalize entries to 4-tuples before retry
        fixed = [tuple(e[:2]) + tuple(e[2:4]) for e in caches_iterable if len(e) >= 4]
        cache = EncoderDecoderCache(fixed)
    else:
        raise

When it happens

Trigger: Passing a single iterable of per-layer cache tuples to EncoderDecoderCache where some tuple has e.g. 2, 3, or 5 elements — common when materializing caches from distributed (dp/ddp) gathered state or from legacy cache dumps with an unexpected layout.

Common situations: Migrating old DDP-era checkpoints/caches that stored only key/value per layer without sliding-window tensors and also dropped the cross-attn half; hand-building the iterable and mis-slicing; version upgrades that added the sliding tensor (6-tuple) where older code assumed 4.

Related errors

• Expected 1 or 2 arguments, got {len(caches)}
• `axis_value` for `HQQ` backend has to be one of [`0`, `1`] b
• You can construct a Cache either from a list `layers` of all
• You should provide exactly one of `layers` or `layer_class_t
• Cannot call `update_conv_state` on a non-LinearAttention lay

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/46bd8cb5fe97029c. Report an issue: GitHub.