{"record":{"id":"665e859be5fb80b3","repo":"huggingface/transformers","slug":"axis-key-for-quanto-backend-has-to-be-one-of","errorCode":null,"errorMessage":"`axis_key` for `quanto` backend has to be one of [`0`, `-1`] but got {self.axis_key}","messagePattern":"`axis_key` for `quanto` backend has to be one of \\[`0`, `-1`\\] but got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/cache_utils.py","lineNumber":808,"sourceCode":"\n        # We need to import quanto here to avoid circular imports due to optimum/quanto/models/transformers_models.py\n        if not is_optimum_quanto_available():\n            raise ImportError(\n                \"You need to install optimum-quanto in order to use KV cache quantization with optimum-quanto \"\n                \"backend. Please install it via  with `pip install optimum-quanto`\"\n            )\n        elif is_quanto_greater(\"0.2.5\", accept_dev=True):\n            from optimum.quanto import MaxOptimizer, qint2, qint4\n        else:\n            raise ImportError(\n                \"You need optimum-quanto package version to be greater or equal than 0.2.5 to use `QuantoQuantizedLayer`. \"\n            )\n\n        if self.nbits not in [2, 4]:\n            raise ValueError(f\"`nbits` for `quanto` backend has to be one of [`2`, `4`] but got {self.nbits}\")\n\n        if self.axis_key not in [0, -1]:\n            raise ValueError(f\"`axis_key` for `quanto` backend has to be one of [`0`, `-1`] but got {self.axis_key}\")\n\n        if self.axis_value not in [0, -1]:\n            raise ValueError(\n                f\"`axis_value` for `quanto` backend has to be one of [`0`, `-1`] but got {self.axis_value}\"\n            )\n\n        self.qtype = qint4 if self.nbits == 4 else qint2\n        self.optimizer = MaxOptimizer()  # hardcode as it's the only one for per-channel quantization\n\n    def _quantize(self, tensor, axis):\n        from optimum.quanto import quantize_weight\n\n        scale, zeropoint = self.optimizer(tensor, self.qtype, axis, self.q_group_size)\n        qtensor = quantize_weight(tensor, self.qtype, axis, scale, zeropoint, self.q_group_size)\n        return qtensor\n\n    def _dequantize(self, qtensor):\n        return qtensor.dequantize()","sourceCodeStart":790,"sourceCodeEnd":826,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cache_utils.py#L790-L826","documentation":"ValueError in QuantoQuantizedLayer.__init__ validating axis_key: quanto's per-channel quantization supports key tensors quantized along axis 0 (per-token/head groups) or -1 (per-channel along the hidden dim) only. Any other axis (1, 2, positive axes other than 0) is rejected because optimum.quanto's MaxOptimizer/quantize_weight cannot handle it for the KV use-case.","triggerScenarios":"QuantoQuantizedCacheConfig(axis_key=1) (or 2) with backend='quanto'; copying axis values from HQQ configs, which use a different convention (HQQ allows 0/1); passing default axis values of another cache class.","commonSituations":"Switching backends while keeping axis settings; porting from torch-ao quantized cache configs whose axis semantics differ; experimentation with per-head quantization axes.","solutions":["Set axis_key to 0 or -1 for the quanto backend","If migrating from HQQ, translate axes: HQQ 0 -> quanto 0, HQQ 1 -> quanto -1 (channel axis)","Leave axis_key at the config default unless you have a measured reason to change it"],"exampleFix":"# before\nconfig = QuantoQuantizedCacheConfig(nbits=4, axis_key=1, backend='quanto')\n\n# after\nconfig = QuantoQuantizedCacheConfig(nbits=4, axis_key=0, backend='quanto')","handlingStrategy":"validation","validationCode":"assert config.backend != 'quanto' or config.axis_key in (0, -1), 'quanto axis_key must be 0 or -1'","typeGuard":"def is_valid_quanto_axis(axis) -> bool:\n    return axis in (0, -1)","tryCatchPattern":null,"preventionTips":["Remember the backend axis vocabularies: quanto {0, -1} vs HQQ {0, 1}","Translate axes explicitly when porting configs between backends","Leave axis values at defaults unless benchmarks justify a change"],"tags":["quantization","kv-cache","config","validation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}