What causes "`nbits` for `HQQ` backend has to be one of [`1`, `2`, `3`, `4`, `8`] but got {self.nbits}" in huggingface/transformers?

Use nbits in {1, 2, 3, 4, 8} with backend='hqq'

huggingface/transformers · error · ValueError

`nbits` for `HQQ` backend has to be one of [`1`, `2`, `3`, `

Error message

`nbits` for `HQQ` backend has to be one of [`1`, `2`, `3`, `4`, `8`] but got {self.nbits}

What it means

ValueError in HqqQuantizedLayer.__init__ validating nbits for the HQQ backend. HQQ supports 1, 2, 3, 4 and 8 bit quantization for KV caches; other bit-widths (e.g. 16, 5, 6) are rejected here, before HQQQuantizer.quantize is ever invoked, because HQQ has no calibrated kernels for them.

Source

Thrown at src/transformers/cache_utils.py:853

        q_group_size: int = 64,
        residual_length: int = 128,
    ):
        super().__init__(
            nbits=nbits,
            axis_key=axis_key,
            axis_value=axis_value,
            q_group_size=q_group_size,
            residual_length=residual_length,
        )

        if not is_hqq_available():
            raise ImportError(
                "You need to install `HQQ` in order to use KV cache quantization with HQQ backend. "
                "Please install it via  with `pip install hqq`"
            )

        if self.nbits not in [1, 2, 3, 4, 8]:
            raise ValueError(
                f"`nbits` for `HQQ` backend has to be one of [`1`, `2`, `3`, `4`, `8`] but got {self.nbits}"
            )

        if self.axis_key not in [0, 1]:
            raise ValueError(f"`axis_key` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_key}")

        if self.axis_value not in [0, 1]:
            raise ValueError(f"`axis_value` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_value}")

        self.quantizer = HQQQuantizer

    def _quantize(self, tensor, axis):
        qtensor, meta = self.quantizer.quantize(
            tensor,
            axis=axis,
            device=self.keys.device,
            compute_dtype=self.keys.dtype,
            nbits=self.nbits,

View on GitHub (pinned to a597f97485)

Solutions

Use nbits in {1, 2, 3, 4, 8} with backend='hqq'
If you wanted 2/4-bit with quanto instead, switch backend='quanto'
Validate config-derived values and cast to int before constructing the cache config

Example fix

# before
config = HqqQuantizedCacheConfig(nbits=6, backend='hqq')

# after
config = HqqQuantizedCacheConfig(nbits=8, backend='hqq')

Defensive patterns

Strategy: validation

Validate before calling

assert config.backend == 'hqq' and int(config.nbits) in (1, 2, 3, 4, 8), 'HQQ supports nbits in {1,2,3,4,8}'

Type guard

def is_valid_hqq_nbits(nbits) -> bool:
    return isinstance(nbits, int) and nbits in (1, 2, 3, 4, 8)

Prevention

Validate nbits against the backend-specific allowlist in your config factory
Re-check nbits whenever backend changes
Coerce string values from config files to int before validation

When it happens

Trigger: HqqQuantizedCacheConfig(nbits=5) or nbits=16; copying a quanto config (which only allows 2/4) plus an invalid extra value while switching backend to 'hqq'; nbits passed as a non-int from a YAML config.

Common situations: Experimenting with unusual bit-widths for memory savings; config files shared across teams where nbits was tuned for a different backend; off-by-one typos (e.g. 6 instead of 8).

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/d33e361c574351cc. Report an issue: GitHub.