huggingface/transformers · error · ValueError

`axis_value` for `HQQ` backend has to be one of [`0`, `1`] b

Error message

`axis_value` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_value}

What it means

HQQQuantizedLayer validates its constructor arguments and raises ValueError when axis_value is not 0 or 1. axis_value (like axis_key) selects the tensor dimension along which the HQQ quantizer quantizes the value projections of the KV cache; HQQ only supports axis 0 or 1. The check mirrors the immediately preceding checks for nbits and axis_key in the same __init__.

Source

Thrown at src/transformers/cache_utils.py:861

            residual_length=residual_length,
        )

        if not is_hqq_available():
            raise ImportError(
                "You need to install `HQQ` in order to use KV cache quantization with HQQ backend. "
                "Please install it via  with `pip install hqq`"
            )

        if self.nbits not in [1, 2, 3, 4, 8]:
            raise ValueError(
                f"`nbits` for `HQQ` backend has to be one of [`1`, `2`, `3`, `4`, `8`] but got {self.nbits}"
            )

        if self.axis_key not in [0, 1]:
            raise ValueError(f"`axis_key` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_key}")

        if self.axis_value not in [0, 1]:
            raise ValueError(f"`axis_value` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_value}")

        self.quantizer = HQQQuantizer

    def _quantize(self, tensor, axis):
        qtensor, meta = self.quantizer.quantize(
            tensor,
            axis=axis,
            device=self.keys.device,
            compute_dtype=self.keys.dtype,
            nbits=self.nbits,
            group_size=self.q_group_size,
        )
        meta["compute_dtype"] = self.keys.dtype
        self.quantizer.cuda(qtensor, meta=meta, device=self.keys.device)  # Move to device and cast to dtype
        meta["scale"] = meta["scale"].to(qtensor.device)
        meta["zero"] = meta["zero"].to(qtensor.device)
        return qtensor, meta

View on GitHub (pinned to a597f97485)

Solutions

1. Set axis_value to 0 or 1 (0 quantizes per output-channel column-wise, 1 per input-channel row-wise); for KV cache quantization the common setting is axis_value=0 with axis_key=1 (transposed layout)
2. If the axis came from a config dict/YAML, validate it at load time: assert axis_value in (0, 1) before constructing the cache
3. Double-check axis_key too — the sibling check at the same site rejects invalid axis_key with an analogous message

Example fix

// before
cache = QuantizedCache(config, backend="hqq", axis_value=-1)

// after
cache = QuantizedCache(config, backend="hqq", axis_key=1, axis_value=0)

axis_value = 0
assert axis_key in (0, 1) and axis_value in (0, 1), "HQQ axes must be 0 or 1"
cache = QuantizedCache(config, backend="hqq", axis_key=axis_key, axis_value=axis_value)

def is_valid_hqq_axis(axis: int) -> bool:
    return isinstance(axis, int) and axis in (0, 1)

try:
    cache = QuantizedCache(config, backend="hqq", axis_value=av)
except ValueError as e:
    if "axis_value" in str(e):
        av = 0
        cache = QuantizedCache(config, backend="hqq", axis_value=av)
    else:
        raise

When it happens

Trigger: Constructing QuantizedCache(config, backend="hqq", axis_value=2) or any non-{0,1} value; also instantiating HQQQuantizedLayer directly with an invalid axis_value. QuantizedCache passes axis_value straight into HQQQuantizedLayer, so any bad value surfaces here.

Common situations: Copying a quanto-style config where different axis conventions are used; passing a negative axis (e.g. -1) intending 'last dimension'; programmatic axis selection that produces 2+ for multi-head reshaped tensors.

Related errors

• You can construct a Cache either from a list `layers` of all
• You should provide exactly one of `layers` or `layer_class_t
• Unknown quantization backend `{backend}`
• `QuantizedCache` is only supported for models with only full
• Unsupported quantization method: '{self.quantization}'. Must

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/0d7e77f87933e6e3. Report an issue: GitHub.