huggingface/transformers · error · ValueError

`QuantizedCache` is only supported for models with only full

Error message

`QuantizedCache` is only supported for models with only full attention layers. We found the following invalid layer types: {invalid_layer_types}

What it means

QuantizedCache.__init__ raises ValueError when the model config contains layer types other than 'full_attention' (e.g. sliding_attention, linear_attention). Quantized KV-cache layers assume standard KV projections present only in full attention layers, so hybrid architectures are rejected with the offending types listed.

Source

Thrown at src/transformers/cache_utils.py:1929

        config: PreTrainedConfig,
        nbits: int = 4,
        axis_key: int = 0,
        axis_value: int = 0,
        q_group_size: int = 64,
        residual_length: int = 128,
    ):
        if backend == "quanto":
            layer_class = QuantoQuantizedLayer
        elif backend == "hqq":
            layer_class = HQQQuantizedLayer
        else:
            raise ValueError(f"Unknown quantization backend `{backend}`")

        config = config.get_text_config(decoder=True)
        layer_types, _ = get_layer_types_and_kwargs(config)
        invalid_layer_types = set(layer_types) - {"full_attention"}
        if len(invalid_layer_types) > 0:
            raise ValueError(
                "`QuantizedCache` is only supported for models with only full attention layers. We found the following invalid layer "
                f"types: {invalid_layer_types}"
            )
        layers = [
            layer_class(nbits, axis_key, axis_value, q_group_size, residual_length)
            for _ in range(config.num_hidden_layers)
        ]
        super().__init__(layers=layers)


class EncoderDecoderCache(Cache):
    """
    Base, abstract class for all encoder-decoder caches. Can be used to hold combinations of self-attention and
    cross-attention caches.

    See `Cache` for details on common methods that are implemented by all cache classes.

    Args:

View on GitHub (pinned to a597f97485)

Solutions

Use a standard DynamicCache (or model-supported alternative) for models with sliding or linear attention layers
Check config.get_text_config(decoder=True).layer_types before attempting QuantizedCache; require all entries to be 'full_attention'
Pick a model variant with full attention only if quantized KV cache is a hard requirement

Example fix

# before
cache = QuantizedCache(config, backend="quanto")  # Gemma-3: sliding_attention present

# after
cfg = config.get_text_config(decoder=True)
if set(get_layer_types_and_kwargs(cfg)[0]) <= {"full_attention"}:
    cache = QuantizedCache(config, backend="quanto")
else:
    cache = DynamicCache()

Defensive patterns

Strategy: validation

Validate before calling

from transformers.cache_utils import get_layer_types_and_kwargs

cfg = config.get_text_config(decoder=True)
layer_types, _ = get_layer_types_and_kwargs(cfg)
if set(layer_types) <= {"full_attention"}:
    cache = QuantizedCache(config, backend="hqq")
else:
    cache = DynamicCache()

Type guard

from transformers.cache_utils import get_layer_types_and_kwargs

def supports_quantized_cache(config) -> bool:
    cfg = config.get_text_config(decoder=True)
    layer_types, _ = get_layer_types_and_kwargs(cfg)
    return set(layer_types) <= {"full_attention"}

Try / catch

try:
    cache = QuantizedCache(config, backend="quanto")
except ValueError as e:
    if "only supported for models with only full attention" in str(e):
        cache = DynamicCache()
    else:
        raise

Prevention

Check config.layer_types before wiring QuantizedCache; sliding or linear attention layers disqualify it
Pin your model choice when KV-cache quantization is required: full-attention architectures only
Surface this check early at setup time, not mid-generation

When it happens

Trigger: QuantizedCache(config, backend=...) with a config whose layer_types include sliding attention (Gemma-2/3, Cohere2) or linear attention (Qwen3-Next, Falcon-H1, Mamba hybrids).

Common situations: Applying KV-cache quantization to modern hybrid or sliding-window models; reusing a working QuantizedCache setup from Llama on a newer architecture without checking its layer types.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/e4caa7b492eb4ccd. Report an issue: GitHub.