{"record":{"id":"da99dbf0f46c6720","repo":"huggingface/transformers","slug":"unknown-quantization-backend-backend","errorCode":null,"errorMessage":"Unknown quantization backend `{backend}`","messagePattern":"Unknown quantization backend `(.+?)`","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/cache_utils.py","lineNumber":1923,"sourceCode":"            Maximum capacity for the original precision cache\n    \"\"\"\n\n    def __init__(\n        self,\n        backend: str,\n        config: PreTrainedConfig,\n        nbits: int = 4,\n        axis_key: int = 0,\n        axis_value: int = 0,\n        q_group_size: int = 64,\n        residual_length: int = 128,\n    ):\n        if backend == \"quanto\":\n            layer_class = QuantoQuantizedLayer\n        elif backend == \"hqq\":\n            layer_class = HQQQuantizedLayer\n        else:\n            raise ValueError(f\"Unknown quantization backend `{backend}`\")\n\n        config = config.get_text_config(decoder=True)\n        layer_types, _ = get_layer_types_and_kwargs(config)\n        invalid_layer_types = set(layer_types) - {\"full_attention\"}\n        if len(invalid_layer_types) > 0:\n            raise ValueError(\n                \"`QuantizedCache` is only supported for models with only full attention layers. We found the following invalid layer \"\n                f\"types: {invalid_layer_types}\"\n            )\n        layers = [\n            layer_class(nbits, axis_key, axis_value, q_group_size, residual_length)\n            for _ in range(config.num_hidden_layers)\n        ]\n        super().__init__(layers=layers)\n\n\nclass EncoderDecoderCache(Cache):\n    \"\"\"","sourceCodeStart":1905,"sourceCodeEnd":1941,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cache_utils.py#L1905-L1941","documentation":"QuantizedCache.__init__ raises ValueError when backend is neither 'quanto' nor 'hqq'. These are the only two KV-cache quantization backends wired into the constructor; anything else fails before any layer is built.","triggerScenarios":"QuantizedCache(config, backend='bitsandbytes') or similar; passing a backend string with different casing/whitespace; passing None or an empty string.","commonSituations":"Confusing weight-quantization backend names (bnb, gguf, awq) with KV-cache quantization backends (quanto, hqq); typos; config-driven backend names that drift from the supported set.","solutions":["Use backend='quanto' or backend='hqq' (optimum-quanto or hqq must be installed respectively)","Normalize/validate the backend string at config load time: backend in {'quanto', 'hqq'}","For bitsandbytes-style quantization, note it applies to model weights (BitsAndBytesConfig), not the KV cache via this API"],"exampleFix":"# before\ncache = QuantizedCache(config, backend=\"bnb\")\n\n# after\ncache = QuantizedCache(config, backend=\"hqq\", nbits=4, axis_key=1, axis_value=0)","handlingStrategy":"validation","validationCode":"SUPPORTED = {\"quanto\", \"hqq\"}\nassert backend in SUPPORTED, f\"backend must be one of {SUPPORTED}, got {backend!r}\"\ncache = QuantizedCache(config, backend=backend)","typeGuard":"def is_supported_kv_quant_backend(backend: str) -> bool:\n    return isinstance(backend, str) and backend.lower() in {\"quanto\", \"hqq\"}","tryCatchPattern":"try:\n    cache = QuantizedCache(config, backend=backend)\nexcept ValueError as e:\n    if \"Unknown quantization backend\" in str(e):\n        cache = QuantizedCache(config, backend=\"hqq\")  # explicit fallback choice\n    else:\n        raise","preventionTips":["Keep the backend name in one config field validated against {'quanto','hqq'} at load","Install the matching package (pip install optimum-quanto or hqq) — a separate ImportError fires if missing","Do not conflate weight-quantization backends with KV-cache quantization backends"],"tags":["cache","quantization","backend","valueerror"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}