What causes "`axis_key` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_key}" in huggingface/transformers?

Use axis_key in {0, 1} for the HQQ backend

huggingface/transformers · error · ValueError

`axis_key` for `HQQ` backend has to be one of [`0`, `1`] but

Error message

`axis_key` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_key}

What it means

ValueError in HqqQuantizedLayer.__init__ validating axis_key for the HQQ backend. HQQ quantizes along axis 0 (rows, i.e. per token group) or axis 1 (channels/heads); -1 and other axes are not supported by HQQ's quantizer and are rejected at layer construction. Note the convention clash with quanto, which accepts {0, -1} — a direct config port between backends triggers this.

Source

Thrown at src/transformers/cache_utils.py:858

            axis_key=axis_key,
            axis_value=axis_value,
            q_group_size=q_group_size,
            residual_length=residual_length,
        )

        if not is_hqq_available():
            raise ImportError(
                "You need to install `HQQ` in order to use KV cache quantization with HQQ backend. "
                "Please install it via  with `pip install hqq`"
            )

        if self.nbits not in [1, 2, 3, 4, 8]:
            raise ValueError(
                f"`nbits` for `HQQ` backend has to be one of [`1`, `2`, `3`, `4`, `8`] but got {self.nbits}"
            )

        if self.axis_key not in [0, 1]:
            raise ValueError(f"`axis_key` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_key}")

        if self.axis_value not in [0, 1]:
            raise ValueError(f"`axis_value` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_value}")

        self.quantizer = HQQQuantizer

    def _quantize(self, tensor, axis):
        qtensor, meta = self.quantizer.quantize(
            tensor,
            axis=axis,
            device=self.keys.device,
            compute_dtype=self.keys.dtype,
            nbits=self.nbits,
            group_size=self.q_group_size,
        )
        meta["compute_dtype"] = self.keys.dtype
        self.quantizer.cuda(qtensor, meta=meta, device=self.keys.device)  # Move to device and cast to dtype
        meta["scale"] = meta["scale"].to(qtensor.device)

View on GitHub (pinned to a597f97485)

Solutions

Use axis_key in {0, 1} for the HQQ backend
When porting from quanto: quanto -1 (channel) -> HQQ 1; quanto 0 -> HQQ 0
Keep axis_key and axis_value both within {0, 1} for HQQ

Example fix

# before (ported from quanto)
config = HqqQuantizedCacheConfig(nbits=8, axis_key=-1, backend='hqq')

# after
config = HqqQuantizedCacheConfig(nbits=8, axis_key=1, backend='hqq')

Defensive patterns

Strategy: validation

Validate before calling

assert config.backend != 'hqq' or config.axis_key in (0, 1), 'HQQ axis_key must be 0 or 1'

Type guard

def is_valid_hqq_axis(axis) -> bool:
    return axis in (0, 1)

Prevention

Use the mapping quanto -1 -> HQQ 1 when porting configs
Validate both axes with the backend's allowlist before constructing the cache
Keep backend-specific config builders so axes and nbits are always paired correctly

When it happens

Trigger: HqqQuantizedCacheConfig(axis_key=-1) (a valid quanto axis) with backend='hqq'; axis_key=2; configs migrated from QuantoQuantizedCacheConfig without translating the axis.

Common situations: Backend switching without axis translation; docs/examples mixing the two backends' conventions; symmetric copy-paste where axis_value was translated but axis_key was not.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/0cea9499bfee8c66. Report an issue: GitHub.