huggingface/transformers · error · ImportError

You need to install optimum-quanto in order to use KV cache

Error message

You need to install optimum-quanto in order to use KV cache quantization with optimum-quanto backend. Please install it via  with `pip install optimum-quanto`

What it means

ImportError raised in QuantoQuantizedLayer.__init__ when KV-cache quantization is requested with backend='quanto' (e.g. CacheConfig like QuantoQuantizedCacheConfig) but the optimum-quanto package is not installed. The import is done lazily inside the layer constructor to avoid a hard dependency and circular imports, so the failure appears at cache construction time, not at import time of transformers.

Source

Thrown at src/transformers/cache_utils.py:793

    def __init__(
        self,
        nbits: int = 4,
        axis_key: int = 0,
        axis_value: int = 0,
        q_group_size: int = 64,
        residual_length: int = 128,
    ):
        super().__init__(
            nbits=nbits,
            axis_key=axis_key,
            axis_value=axis_value,
            q_group_size=q_group_size,
            residual_length=residual_length,
        )

        # We need to import quanto here to avoid circular imports due to optimum/quanto/models/transformers_models.py
        if not is_optimum_quanto_available():
            raise ImportError(
                "You need to install optimum-quanto in order to use KV cache quantization with optimum-quanto "
                "backend. Please install it via  with `pip install optimum-quanto`"
            )
        elif is_quanto_greater("0.2.5", accept_dev=True):
            from optimum.quanto import MaxOptimizer, qint2, qint4
        else:
            raise ImportError(
                "You need optimum-quanto package version to be greater or equal than 0.2.5 to use `QuantoQuantizedLayer`. "
            )

        if self.nbits not in [2, 4]:
            raise ValueError(f"`nbits` for `quanto` backend has to be one of [`2`, `4`] but got {self.nbits}")

        if self.axis_key not in [0, -1]:
            raise ValueError(f"`axis_key` for `quanto` backend has to be one of [`0`, `-1`] but got {self.axis_key}")

        if self.axis_value not in [0, -1]:
            raise ValueError(

View on GitHub (pinned to a597f97485)

Solutions

  1. pip install optimum-quanto
  2. Pin a compatible version: pip install 'optimum-quanto>=0.2.5'
  3. If quantization is optional at runtime, guard with transformers.utils.is_optimum_quanto_available() and fall back to an unquantized cache

Example fix

# before
config = QuantoQuantizedCacheConfig(nbits=4, backend='quanto')
cache = DynamicCache.from_config(config)  # ImportError

# after
# shell: pip install optimum-quanto
config = QuantoQuantizedCacheConfig(nbits=4, backend='quanto')
cache = DynamicCache.from_config(config)
Defensive patterns

Strategy: validation

Validate before calling

from transformers.utils.import_utils import is_optimum_quanto_available
if not is_optimum_quanto_available():
    raise SystemExit("This script needs KV quantization: pip install optimum-quanto")

Try / catch

try:
    cache = DynamicCache.from_config(quant_config)
except ImportError as e:
    if 'optimum-quanto' in str(e):
        cache = DynamicCache()  # unquantized fallback
    else:
        raise

Prevention

When it happens

Trigger: QuantoQuantizedCacheConfig(...)/DynamicCache.from_config with quantization backend 'quanto' while optimum-quanto is absent from the environment; running inference code that worked in another env; CI images without the quantization extras.

Common situations: Copying KV-quantization examples into a minimal environment; deploying to production images built from a bare transformers install; upgrading environments and dropping the optional dependency.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/997a8e5bf8c3e20c. Report an issue: GitHub.