huggingface/transformers · error · ValueError

You can construct a Cache either from a list `layers` of all

Error message

You can construct a Cache either from a list `layers` of all the predefined `CacheLayer`, or from a `layer_class_to_replicate`, in which case the Cache will append a new layer corresponding to `layer_class_to_replicate` for each new call to `update` with an idx not already in the Cache.

What it means

Cache.__init__ raises ValueError when both layers and layer_class_to_replicate are provided. The Cache base class supports two mutually exclusive construction modes: pre-built layers (static caches like StaticCache/QuantizedCache) or lazy replication of a class per layer (DynamicCache), never both.

Source

Thrown at src/transformers/cache_utils.py:1290

            Only used if `layers` is omitted (`None`), in which case it will be used as the base class for each layer,
            and the layers will be added lazily as soon as `update` is called with a `layer_idx` greater than the current
            list of layers.
        offloading (`bool`, *optional*, defaults to `False`):
            Whether to perform offloading of the layers to `cpu`, to save GPU memory.
        offload_only_non_sliding (`bool`, *optional*, defaults to `True`):
            If `offloading` is `True`, this further decides if only the non-sliding layers will be offloaded (because
            usually the sliding layers are small in size, so there is no need to offload them, and skipping it is faster).
    """

    def __init__(
        self,
        layers: list[CacheLayerMixin | LinearAttentionCacheLayerMixin] | None = None,
        layer_class_to_replicate: type[CacheLayerMixin | LinearAttentionCacheLayerMixin] | None = None,
        offloading: bool = False,
        offload_only_non_sliding: bool = True,
    ):
        if layers is not None and layer_class_to_replicate is not None:
            raise ValueError(
                "You can construct a Cache either from a list `layers` of all the predefined `CacheLayer`, or from a "
                "`layer_class_to_replicate`, in which case the Cache will append a new layer corresponding to "
                "`layer_class_to_replicate` for each new call to `update` with an idx not already in the Cache."
            )
        if layers is None and layer_class_to_replicate is None:
            raise ValueError(
                "You should provide exactly one of `layers` or `layer_class_to_replicate` to initialize a Cache."
            )
        self.layers = layers if layers is not None else []
        self.layer_class_to_replicate = layer_class_to_replicate
        self.offloading = offloading
        if self.offloading:
            self.only_non_sliding = offload_only_non_sliding
            self.prefetch_stream = torch.Stream() if _is_torch_greater_or_equal_than_2_7 else torch.cuda.Stream()

    def __repr__(self):
        return f"{self.__class__.__name__}(layers={self.layers})"

View on GitHub (pinned to a597f97485)

Solutions

Pass exactly one: either layers=[CacheLayer...] (pre-initialized, e.g. for static/quantized caches) or layer_class_to_replicate=SomeLayerClass (lazy, e.g. DynamicCache)
If you want a DynamicCache pre-filled with tensors, pass layers built from your tensors instead of layer_class_to_replicate
Check subclass __init__ defaults that may silently fill in one of the two arguments

Example fix

# before
cache = DynamicCache(layers=prefilled_layers, layer_class_to_replicate=DynamicCacheLayer)

# after
cache = DynamicCache(layers=prefilled_layers)

Defensive patterns

Strategy: validation

Validate before calling

assert not (layers is not None and layer_class_to_replicate is not None), (
    "pass exactly one of layers / layer_class_to_replicate"
)
cache = Cache(layers=layers) if layers is not None else Cache(layer_class_to_replicate=DynamicCacheLayer)

Try / catch

try:
    cache = Cache(layers=layers, layer_class_to_replicate=cls)
except ValueError:
    cache = Cache(layers=layers)  # prefer pre-built layers when both were supplied

Prevention

In subclasses, make layer_class_to_replicate a fixed class attribute instead of an __init__ parameter
Pass pre-built layers for static layouts and the replicate class for dynamic ones — never mix

When it happens

Trigger: Calling Cache(layers=[...], layer_class_to_replicate=DynamicCacheLayer) or a subclass whose __init__ forwards both arguments, e.g. DynamicCache(layers=my_layers, layer_class_to_replicate=...).

Common situations: Subclassing Cache or DynamicCache and passing through a user-supplied layer list while a default layer_class_to_replicate is also set; migrating code from older transformers where DynamicCache() took no arguments and then adding preloaded layers (e.g. from from_legacy_cache-style data).

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/5d25db6733dc8193. Report an issue: GitHub.