{"record":{"id":"76e10fdf09f42ef6","repo":"huggingface/transformers","slug":"nbits-for-quanto-backend-has-to-be-one-of-2","errorCode":null,"errorMessage":"`nbits` for `quanto` backend has to be one of [`2`, `4`] but got {self.nbits}","messagePattern":"`nbits` for `quanto` backend has to be one of \\[`2`, `4`\\] but got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/cache_utils.py","lineNumber":805,"sourceCode":"            q_group_size=q_group_size,\n            residual_length=residual_length,\n        )\n\n        # We need to import quanto here to avoid circular imports due to optimum/quanto/models/transformers_models.py\n        if not is_optimum_quanto_available():\n            raise ImportError(\n                \"You need to install optimum-quanto in order to use KV cache quantization with optimum-quanto \"\n                \"backend. Please install it via  with `pip install optimum-quanto`\"\n            )\n        elif is_quanto_greater(\"0.2.5\", accept_dev=True):\n            from optimum.quanto import MaxOptimizer, qint2, qint4\n        else:\n            raise ImportError(\n                \"You need optimum-quanto package version to be greater or equal than 0.2.5 to use `QuantoQuantizedLayer`. \"\n            )\n\n        if self.nbits not in [2, 4]:\n            raise ValueError(f\"`nbits` for `quanto` backend has to be one of [`2`, `4`] but got {self.nbits}\")\n\n        if self.axis_key not in [0, -1]:\n            raise ValueError(f\"`axis_key` for `quanto` backend has to be one of [`0`, `-1`] but got {self.axis_key}\")\n\n        if self.axis_value not in [0, -1]:\n            raise ValueError(\n                f\"`axis_value` for `quanto` backend has to be one of [`0`, `-1`] but got {self.axis_value}\"\n            )\n\n        self.qtype = qint4 if self.nbits == 4 else qint2\n        self.optimizer = MaxOptimizer()  # hardcode as it's the only one for per-channel quantization\n\n    def _quantize(self, tensor, axis):\n        from optimum.quanto import quantize_weight\n\n        scale, zeropoint = self.optimizer(tensor, self.qtype, axis, self.q_group_size)\n        qtensor = quantize_weight(tensor, self.qtype, axis, scale, zeropoint, self.q_group_size)\n        return qtensor","sourceCodeStart":787,"sourceCodeEnd":823,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cache_utils.py#L787-L823","documentation":"ValueError in QuantoQuantizedLayer.__init__ validating nbits: the optimum-quanto backend only supports 2-bit and 4-bit KV-cache quantization (qint2/qint4 are the only qtypes wired up). Any other value — 8, 3, 1, 16 — is rejected at cache construction time before any tensor is quantized.","triggerScenarios":"QuantoQuantizedCacheConfig(nbits=8, backend='quanto'), nbits=3, or copying an HQQ-style config (which allows 1/2/3/4/8) and switching backend='quanto' without adjusting nbits; also passing nbits as a string ('4') raises the same error.","commonSituations":"Migrating configs between HQQ and quanto backends; assuming 8-bit is supported because other transformers quantizers offer it; YAML/JSON configs where nbits defaults to something other than 2/4.","solutions":["Use nbits=2 or nbits=4 with backend='quanto'","If you need 8-bit or 3-bit, switch backend to 'hqq' (supports 1/2/3/4/8)","Cast nbits to int if it comes from a config file as a string"],"exampleFix":"# before\nconfig = QuantoQuantizedCacheConfig(nbits=8, backend='quanto')\n\n# after\nconfig = QuantoQuantizedCacheConfig(nbits=4, backend='quanto')\n# or use HQQ for 8-bit:\nconfig = HqqQuantizedCacheConfig(nbits=8, backend='hqq')","handlingStrategy":"validation","validationCode":"assert config.backend == 'quanto' and int(config.nbits) in (2, 4), 'quanto backend supports only 2/4 bits'","typeGuard":"def is_valid_quanto_nbits(nbits) -> bool:\n    return isinstance(nbits, int) and nbits in (2, 4)","tryCatchPattern":null,"preventionTips":["Keep a per-backend table of allowed nbits (quanto: 2/4; hqq: 1/2/3/4/8) in your config builder","When switching backends, re-validate nbits and axes together","Cast nbits to int when loading configs from YAML/JSON"],"tags":["quantization","kv-cache","config","validation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}