huggingface/transformers · error · ImportError

You need to install `HQQ` in order to use KV cache quantizat

Error message

You need to install `HQQ` in order to use KV cache quantization with HQQ backend. Please install it via  with `pip install hqq`

What it means

ImportError raised in HqqQuantizedLayer.__init__ when KV-cache quantization is requested with backend='hqq' (HqqQuantizedCacheConfig) but the hqq package is missing. Like quanto, HQQ is an optional dependency imported lazily at cache-construction time, so the failure surfaces when DynamicCache.from_config(config) instantiates the quantized layers, not at transformers import.

Source

Thrown at src/transformers/cache_utils.py:847

class HQQQuantizedLayer(QuantizedLayer):
    def __init__(
        self,
        nbits: int = 4,
        axis_key: int = 0,
        axis_value: int = 0,
        q_group_size: int = 64,
        residual_length: int = 128,
    ):
        super().__init__(
            nbits=nbits,
            axis_key=axis_key,
            axis_value=axis_value,
            q_group_size=q_group_size,
            residual_length=residual_length,
        )

        if not is_hqq_available():
            raise ImportError(
                "You need to install `HQQ` in order to use KV cache quantization with HQQ backend. "
                "Please install it via  with `pip install hqq`"
            )

        if self.nbits not in [1, 2, 3, 4, 8]:
            raise ValueError(
                f"`nbits` for `HQQ` backend has to be one of [`1`, `2`, `3`, `4`, `8`] but got {self.nbits}"
            )

        if self.axis_key not in [0, 1]:
            raise ValueError(f"`axis_key` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_key}")

        if self.axis_value not in [0, 1]:
            raise ValueError(f"`axis_value` for `HQQ` backend has to be one of [`0`, `1`] but got {self.axis_value}")

        self.quantizer = HQQQuantizer

    def _quantize(self, tensor, axis):

View on GitHub (pinned to a597f97485)

Solutions

pip install hqq
Guard at runtime with transformers.utils.import_utils.is_hqq_available() and fall back to DynamicCache
If you cannot add dependencies, use an unquantized cache or the quanto backend if optimum-quanto is present

Example fix

# before
config = HqqQuantizedCacheConfig(nbits=8)
cache = DynamicCache.from_config(config)  # ImportError

# after
# shell: pip install hqq
config = HqqQuantizedCacheConfig(nbits=8)
cache = DynamicCache.from_config(config)

Defensive patterns

Strategy: validation

Validate before calling

from transformers.utils.import_utils import is_hqq_available
if not is_hqq_available():
    raise SystemExit('This script needs HQQ KV quantization: pip install hqq')

Try / catch

try:
    cache = DynamicCache.from_config(quant_config)
except ImportError as e:
    if 'hqq' in str(e):
        cache = DynamicCache()  # unquantized fallback
    else:
        raise

Prevention

Add hqq to requirements for any pipeline using backend='hqq'
Gate with is_hqq_available() so the code degrades gracefully
Bake optional quantization deps into deployment images explicitly

When it happens

Trigger: HqqQuantizedCacheConfig(nbits=8, axis_key=0, axis_value=0) used in an environment without hqq; deploying quantized-cache generation code to a slim production image; fresh venv missing quantization extras.

Common situations: CI/prod environments built from minimal requirements.txt; sharing notebooks that use HQQ KV quantization; uninstalling hqq after an experiment while configs still request it.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/8636057d9f6351db. Report an issue: GitHub.