{"record":{"id":"d0d10082aac2ac1b","repo":"huggingface/transformers","slug":"axis-value-for-quanto-backend-has-to-be-one-of","errorCode":null,"errorMessage":"`axis_value` for `quanto` backend has to be one of [`0`, `-1`] but got {self.axis_value}","messagePattern":"`axis_value` for `quanto` backend has to be one of \\[`0`, `-1`\\] but got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/cache_utils.py","lineNumber":811,"sourceCode":"            raise ImportError(\n                \"You need to install optimum-quanto in order to use KV cache quantization with optimum-quanto \"\n                \"backend. Please install it via  with `pip install optimum-quanto`\"\n            )\n        elif is_quanto_greater(\"0.2.5\", accept_dev=True):\n            from optimum.quanto import MaxOptimizer, qint2, qint4\n        else:\n            raise ImportError(\n                \"You need optimum-quanto package version to be greater or equal than 0.2.5 to use `QuantoQuantizedLayer`. \"\n            )\n\n        if self.nbits not in [2, 4]:\n            raise ValueError(f\"`nbits` for `quanto` backend has to be one of [`2`, `4`] but got {self.nbits}\")\n\n        if self.axis_key not in [0, -1]:\n            raise ValueError(f\"`axis_key` for `quanto` backend has to be one of [`0`, `-1`] but got {self.axis_key}\")\n\n        if self.axis_value not in [0, -1]:\n            raise ValueError(\n                f\"`axis_value` for `quanto` backend has to be one of [`0`, `-1`] but got {self.axis_value}\"\n            )\n\n        self.qtype = qint4 if self.nbits == 4 else qint2\n        self.optimizer = MaxOptimizer()  # hardcode as it's the only one for per-channel quantization\n\n    def _quantize(self, tensor, axis):\n        from optimum.quanto import quantize_weight\n\n        scale, zeropoint = self.optimizer(tensor, self.qtype, axis, self.q_group_size)\n        qtensor = quantize_weight(tensor, self.qtype, axis, scale, zeropoint, self.q_group_size)\n        return qtensor\n\n    def _dequantize(self, qtensor):\n        return qtensor.dequantize()\n\n\nclass HQQQuantizedLayer(QuantizedLayer):","sourceCodeStart":793,"sourceCodeEnd":829,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cache_utils.py#L793-L829","documentation":"ValueError in QuantoQuantizedLayer.__init__ validating axis_value, the quantization axis for value tensors. Symmetric with axis_key: the quanto backend only accepts 0 (per row/token group) or -1 (per channel). Values like 1 or 2 are rejected at cache-layer construction before quantization runs.","triggerScenarios":"QuantoQuantizedCacheConfig(axis_value=1) with backend='quanto'; configs ported from HQQQuantizedCacheConfig where axis conventions are 0/1; asymmetric configs where the user changed axis_key but forgot axis_value or vice versa.","commonSituations":"Backend migration without axis translation; copy-pasted quantization configs from blog posts targeting a different backend; defaults overridden globally in a project's config factory.","solutions":["Set axis_value to 0 or -1 for the quanto backend","Translate HQQ axes when migrating: HQQ 1 (channel) -> quanto -1","Keep axis_key/axis_value consistent unless you intentionally want asymmetric quantization"],"exampleFix":"# before\nconfig = QuantoQuantizedCacheConfig(nbits=4, axis_value=1, backend='quanto')\n\n# after\nconfig = QuantoQuantizedCacheConfig(nbits=4, axis_value=-1, backend='quanto')","handlingStrategy":"validation","validationCode":"assert config.backend != 'quanto' or config.axis_value in (0, -1), 'quanto axis_value must be 0 or -1'","typeGuard":"def is_valid_quanto_axis(axis) -> bool:\n    return axis in (0, -1)","tryCatchPattern":null,"preventionTips":["Validate axis_key and axis_value together, with the same backend-specific rule set","Document your axis choices next to the config so future edits stay in range","Add a unit test that constructs your cache config to catch invalid axes before deployment"],"tags":["quantization","kv-cache","config","validation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}