{"record":{"id":"d33e361c574351cc","repo":"huggingface/transformers","slug":"nbits-for-hqq-backend-has-to-be-one-of-1","errorCode":null,"errorMessage":"`nbits` for `HQQ` backend has to be one of [`1`, `2`, `3`, `4`, `8`] but got {self.nbits}","messagePattern":"`nbits` for `HQQ` backend has to be one of \\[`1`, `2`, `3`, `4`, `8`\\] but got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/cache_utils.py","lineNumber":853,"sourceCode":"        q_group_size: int = 64,\n        residual_length: int = 128,\n    ):\n        super().__init__(\n            nbits=nbits,\n            axis_key=axis_key,\n            axis_value=axis_value,\n            q_group_size=q_group_size,\n            residual_length=residual_length,\n        )\n\n        if not is_hqq_available():\n            raise ImportError(\n                \"You need to install `HQQ` in order to use KV cache quantization with HQQ backend. \"\n                \"Please install it via  with `pip install hqq`\"\n            )\n\n        if self.nbits not in [1, 2, 3, 4, 8]:\n            raise ValueError(\n                f\"`nbits` for `HQQ` backend has to be one of [`1`, `2`, `3`, `4`, `8`] but got {self.nbits}\"\n            )\n\n        if self.axis_key not in [0, 1]:\n            raise ValueError(f\"`axis_key` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_key}\")\n\n        if self.axis_value not in [0, 1]:\n            raise ValueError(f\"`axis_value` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_value}\")\n\n        self.quantizer = HQQQuantizer\n\n    def _quantize(self, tensor, axis):\n        qtensor, meta = self.quantizer.quantize(\n            tensor,\n            axis=axis,\n            device=self.keys.device,\n            compute_dtype=self.keys.dtype,\n            nbits=self.nbits,","sourceCodeStart":835,"sourceCodeEnd":871,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cache_utils.py#L835-L871","documentation":"ValueError in HqqQuantizedLayer.__init__ validating nbits for the HQQ backend. HQQ supports 1, 2, 3, 4 and 8 bit quantization for KV caches; other bit-widths (e.g. 16, 5, 6) are rejected here, before HQQQuantizer.quantize is ever invoked, because HQQ has no calibrated kernels for them.","triggerScenarios":"HqqQuantizedCacheConfig(nbits=5) or nbits=16; copying a quanto config (which only allows 2/4) plus an invalid extra value while switching backend to 'hqq'; nbits passed as a non-int from a YAML config.","commonSituations":"Experimenting with unusual bit-widths for memory savings; config files shared across teams where nbits was tuned for a different backend; off-by-one typos (e.g. 6 instead of 8).","solutions":["Use nbits in {1, 2, 3, 4, 8} with backend='hqq'","If you wanted 2/4-bit with quanto instead, switch backend='quanto'","Validate config-derived values and cast to int before constructing the cache config"],"exampleFix":"# before\nconfig = HqqQuantizedCacheConfig(nbits=6, backend='hqq')\n\n# after\nconfig = HqqQuantizedCacheConfig(nbits=8, backend='hqq')","handlingStrategy":"validation","validationCode":"assert config.backend == 'hqq' and int(config.nbits) in (1, 2, 3, 4, 8), 'HQQ supports nbits in {1,2,3,4,8}'","typeGuard":"def is_valid_hqq_nbits(nbits) -> bool:\n    return isinstance(nbits, int) and nbits in (1, 2, 3, 4, 8)","tryCatchPattern":null,"preventionTips":["Validate nbits against the backend-specific allowlist in your config factory","Re-check nbits whenever backend changes","Coerce string values from config files to int before validation"],"tags":["quantization","kv-cache","config","validation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}