{"record":{"id":"997a8e5bf8c3e20c","repo":"huggingface/transformers","slug":"you-need-to-install-optimum-quanto-in-order-to-use","errorCode":null,"errorMessage":"You need to install optimum-quanto in order to use KV cache quantization with optimum-quanto backend. Please install it via  with `pip install optimum-quanto`","messagePattern":"You need to install optimum-quanto in order to use KV cache quantization with optimum-quanto backend\\. Please install it via  with `pip install optimum-quanto`","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"src/transformers/cache_utils.py","lineNumber":793,"sourceCode":"    def __init__(\n        self,\n        nbits: int = 4,\n        axis_key: int = 0,\n        axis_value: int = 0,\n        q_group_size: int = 64,\n        residual_length: int = 128,\n    ):\n        super().__init__(\n            nbits=nbits,\n            axis_key=axis_key,\n            axis_value=axis_value,\n            q_group_size=q_group_size,\n            residual_length=residual_length,\n        )\n\n        # We need to import quanto here to avoid circular imports due to optimum/quanto/models/transformers_models.py\n        if not is_optimum_quanto_available():\n            raise ImportError(\n                \"You need to install optimum-quanto in order to use KV cache quantization with optimum-quanto \"\n                \"backend. Please install it via  with `pip install optimum-quanto`\"\n            )\n        elif is_quanto_greater(\"0.2.5\", accept_dev=True):\n            from optimum.quanto import MaxOptimizer, qint2, qint4\n        else:\n            raise ImportError(\n                \"You need optimum-quanto package version to be greater or equal than 0.2.5 to use `QuantoQuantizedLayer`. \"\n            )\n\n        if self.nbits not in [2, 4]:\n            raise ValueError(f\"`nbits` for `quanto` backend has to be one of [`2`, `4`] but got {self.nbits}\")\n\n        if self.axis_key not in [0, -1]:\n            raise ValueError(f\"`axis_key` for `quanto` backend has to be one of [`0`, `-1`] but got {self.axis_key}\")\n\n        if self.axis_value not in [0, -1]:\n            raise ValueError(","sourceCodeStart":775,"sourceCodeEnd":811,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cache_utils.py#L775-L811","documentation":"ImportError raised in QuantoQuantizedLayer.__init__ when KV-cache quantization is requested with backend='quanto' (e.g. CacheConfig like QuantoQuantizedCacheConfig) but the optimum-quanto package is not installed. The import is done lazily inside the layer constructor to avoid a hard dependency and circular imports, so the failure appears at cache construction time, not at import time of transformers.","triggerScenarios":"QuantoQuantizedCacheConfig(...)/DynamicCache.from_config with quantization backend 'quanto' while optimum-quanto is absent from the environment; running inference code that worked in another env; CI images without the quantization extras.","commonSituations":"Copying KV-quantization examples into a minimal environment; deploying to production images built from a bare transformers install; upgrading environments and dropping the optional dependency.","solutions":["pip install optimum-quanto","Pin a compatible version: pip install 'optimum-quanto>=0.2.5'","If quantization is optional at runtime, guard with transformers.utils.is_optimum_quanto_available() and fall back to an unquantized cache"],"exampleFix":"# before\nconfig = QuantoQuantizedCacheConfig(nbits=4, backend='quanto')\ncache = DynamicCache.from_config(config)  # ImportError\n\n# after\n# shell: pip install optimum-quanto\nconfig = QuantoQuantizedCacheConfig(nbits=4, backend='quanto')\ncache = DynamicCache.from_config(config)","handlingStrategy":"validation","validationCode":"from transformers.utils.import_utils import is_optimum_quanto_available\nif not is_optimum_quanto_available():\n    raise SystemExit(\"This script needs KV quantization: pip install optimum-quanto\")","typeGuard":null,"tryCatchPattern":"try:\n    cache = DynamicCache.from_config(quant_config)\nexcept ImportError as e:\n    if 'optimum-quanto' in str(e):\n        cache = DynamicCache()  # unquantized fallback\n    else:\n        raise","preventionTips":["Add optimum-quanto to your requirements whenever configs use backend='quanto'","Gate quantized-cache code behind is_optimum_quanto_available()","Include a smoke test that constructs the quantized cache in CI to catch missing deps"],"tags":["dependencies","quantization","kv-cache","import-error"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}