{"record":{"id":"0d7e77f87933e6e3","repo":"huggingface/transformers","slug":"axis-value-for-hqq-backend-has-to-be-one-of","errorCode":null,"errorMessage":"`axis_value` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_value}","messagePattern":"`axis_value` for `HQQ` backend has to be one of \\[`0`, `1`\\] but got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/cache_utils.py","lineNumber":861,"sourceCode":"            residual_length=residual_length,\n        )\n\n        if not is_hqq_available():\n            raise ImportError(\n                \"You need to install `HQQ` in order to use KV cache quantization with HQQ backend. \"\n                \"Please install it via  with `pip install hqq`\"\n            )\n\n        if self.nbits not in [1, 2, 3, 4, 8]:\n            raise ValueError(\n                f\"`nbits` for `HQQ` backend has to be one of [`1`, `2`, `3`, `4`, `8`] but got {self.nbits}\"\n            )\n\n        if self.axis_key not in [0, 1]:\n            raise ValueError(f\"`axis_key` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_key}\")\n\n        if self.axis_value not in [0, 1]:\n            raise ValueError(f\"`axis_value` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_value}\")\n\n        self.quantizer = HQQQuantizer\n\n    def _quantize(self, tensor, axis):\n        qtensor, meta = self.quantizer.quantize(\n            tensor,\n            axis=axis,\n            device=self.keys.device,\n            compute_dtype=self.keys.dtype,\n            nbits=self.nbits,\n            group_size=self.q_group_size,\n        )\n        meta[\"compute_dtype\"] = self.keys.dtype\n        self.quantizer.cuda(qtensor, meta=meta, device=self.keys.device)  # Move to device and cast to dtype\n        meta[\"scale\"] = meta[\"scale\"].to(qtensor.device)\n        meta[\"zero\"] = meta[\"zero\"].to(qtensor.device)\n        return qtensor, meta\n","sourceCodeStart":843,"sourceCodeEnd":879,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cache_utils.py#L843-L879","documentation":"HQQQuantizedLayer validates its constructor arguments and raises ValueError when axis_value is not 0 or 1. axis_value (like axis_key) selects the tensor dimension along which the HQQ quantizer quantizes the value projections of the KV cache; HQQ only supports axis 0 or 1. The check mirrors the immediately preceding checks for nbits and axis_key in the same __init__.","triggerScenarios":"Constructing QuantizedCache(config, backend=\"hqq\", axis_value=2) or any non-{0,1} value; also instantiating HQQQuantizedLayer directly with an invalid axis_value. QuantizedCache passes axis_value straight into HQQQuantizedLayer, so any bad value surfaces here.","commonSituations":"Copying a quanto-style config where different axis conventions are used; passing a negative axis (e.g. -1) intending 'last dimension'; programmatic axis selection that produces 2+ for multi-head reshaped tensors.","solutions":["Set axis_value to 0 or 1 (0 quantizes per output-channel column-wise, 1 per input-channel row-wise); for KV cache quantization the common setting is axis_value=0 with axis_key=1 (transposed layout)","If the axis came from a config dict/YAML, validate it at load time: assert axis_value in (0, 1) before constructing the cache","Double-check axis_key too — the sibling check at the same site rejects invalid axis_key with an analogous message"],"exampleFix":"// before\ncache = QuantizedCache(config, backend=\"hqq\", axis_value=-1)\n\n// after\ncache = QuantizedCache(config, backend=\"hqq\", axis_key=1, axis_value=0)","handlingStrategy":"validation","validationCode":"axis_value = 0\nassert axis_key in (0, 1) and axis_value in (0, 1), \"HQQ axes must be 0 or 1\"\ncache = QuantizedCache(config, backend=\"hqq\", axis_key=axis_key, axis_value=axis_value)","typeGuard":"def is_valid_hqq_axis(axis: int) -> bool:\n    return isinstance(axis, int) and axis in (0, 1)","tryCatchPattern":"try:\n    cache = QuantizedCache(config, backend=\"hqq\", axis_value=av)\nexcept ValueError as e:\n    if \"axis_value\" in str(e):\n        av = 0\n        cache = QuantizedCache(config, backend=\"hqq\", axis_value=av)\n    else:\n        raise","preventionTips":["Hardcode axis_key/axis_value from a reviewed config rather than deriving them programmatically","Validate quantization knobs (backend, nbits, axes, q_group_size) in one place at config load time","Remember nbits must be one of 1/2/3/4/8 — a nearby check rejects others"],"tags":["quantization","hqq","cache","validation","valueerror"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}