{"record":{"id":"e4caa7b492eb4ccd","repo":"huggingface/transformers","slug":"quantizedcache-is-only-supported-for-models-with","errorCode":null,"errorMessage":"`QuantizedCache` is only supported for models with only full attention layers. We found the following invalid layer types: {invalid_layer_types}","messagePattern":"`QuantizedCache` is only supported for models with only full attention layers\\. We found the following invalid layer types: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/cache_utils.py","lineNumber":1929,"sourceCode":"        config: PreTrainedConfig,\n        nbits: int = 4,\n        axis_key: int = 0,\n        axis_value: int = 0,\n        q_group_size: int = 64,\n        residual_length: int = 128,\n    ):\n        if backend == \"quanto\":\n            layer_class = QuantoQuantizedLayer\n        elif backend == \"hqq\":\n            layer_class = HQQQuantizedLayer\n        else:\n            raise ValueError(f\"Unknown quantization backend `{backend}`\")\n\n        config = config.get_text_config(decoder=True)\n        layer_types, _ = get_layer_types_and_kwargs(config)\n        invalid_layer_types = set(layer_types) - {\"full_attention\"}\n        if len(invalid_layer_types) > 0:\n            raise ValueError(\n                \"`QuantizedCache` is only supported for models with only full attention layers. We found the following invalid layer \"\n                f\"types: {invalid_layer_types}\"\n            )\n        layers = [\n            layer_class(nbits, axis_key, axis_value, q_group_size, residual_length)\n            for _ in range(config.num_hidden_layers)\n        ]\n        super().__init__(layers=layers)\n\n\nclass EncoderDecoderCache(Cache):\n    \"\"\"\n    Base, abstract class for all encoder-decoder caches. Can be used to hold combinations of self-attention and\n    cross-attention caches.\n\n    See `Cache` for details on common methods that are implemented by all cache classes.\n\n    Args:","sourceCodeStart":1911,"sourceCodeEnd":1947,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cache_utils.py#L1911-L1947","documentation":"QuantizedCache.__init__ raises ValueError when the model config contains layer types other than 'full_attention' (e.g. sliding_attention, linear_attention). Quantized KV-cache layers assume standard KV projections present only in full attention layers, so hybrid architectures are rejected with the offending types listed.","triggerScenarios":"QuantizedCache(config, backend=...) with a config whose layer_types include sliding attention (Gemma-2/3, Cohere2) or linear attention (Qwen3-Next, Falcon-H1, Mamba hybrids).","commonSituations":"Applying KV-cache quantization to modern hybrid or sliding-window models; reusing a working QuantizedCache setup from Llama on a newer architecture without checking its layer types.","solutions":["Use a standard DynamicCache (or model-supported alternative) for models with sliding or linear attention layers","Check config.get_text_config(decoder=True).layer_types before attempting QuantizedCache; require all entries to be 'full_attention'","Pick a model variant with full attention only if quantized KV cache is a hard requirement"],"exampleFix":"# before\ncache = QuantizedCache(config, backend=\"quanto\")  # Gemma-3: sliding_attention present\n\n# after\ncfg = config.get_text_config(decoder=True)\nif set(get_layer_types_and_kwargs(cfg)[0]) <= {\"full_attention\"}:\n    cache = QuantizedCache(config, backend=\"quanto\")\nelse:\n    cache = DynamicCache()","handlingStrategy":"validation","validationCode":"from transformers.cache_utils import get_layer_types_and_kwargs\n\ncfg = config.get_text_config(decoder=True)\nlayer_types, _ = get_layer_types_and_kwargs(cfg)\nif set(layer_types) <= {\"full_attention\"}:\n    cache = QuantizedCache(config, backend=\"hqq\")\nelse:\n    cache = DynamicCache()","typeGuard":"from transformers.cache_utils import get_layer_types_and_kwargs\n\ndef supports_quantized_cache(config) -> bool:\n    cfg = config.get_text_config(decoder=True)\n    layer_types, _ = get_layer_types_and_kwargs(cfg)\n    return set(layer_types) <= {\"full_attention\"}","tryCatchPattern":"try:\n    cache = QuantizedCache(config, backend=\"quanto\")\nexcept ValueError as e:\n    if \"only supported for models with only full attention\" in str(e):\n        cache = DynamicCache()\n    else:\n        raise","preventionTips":["Check config.layer_types before wiring QuantizedCache; sliding or linear attention layers disqualify it","Pin your model choice when KV-cache quantization is required: full-attention architectures only","Surface this check early at setup time, not mid-generation"],"tags":["cache","quantization","layer-type","sliding-window","hybrid-model","valueerror"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}