huggingface/transformers · error · ValueError

Unknown quantization backend `{backend}`

Error message

Unknown quantization backend `{backend}`

What it means

QuantizedCache.__init__ raises ValueError when backend is neither 'quanto' nor 'hqq'. These are the only two KV-cache quantization backends wired into the constructor; anything else fails before any layer is built.

Source

Thrown at src/transformers/cache_utils.py:1923

            Maximum capacity for the original precision cache
    """

    def __init__(
        self,
        backend: str,
        config: PreTrainedConfig,
        nbits: int = 4,
        axis_key: int = 0,
        axis_value: int = 0,
        q_group_size: int = 64,
        residual_length: int = 128,
    ):
        if backend == "quanto":
            layer_class = QuantoQuantizedLayer
        elif backend == "hqq":
            layer_class = HQQQuantizedLayer
        else:
            raise ValueError(f"Unknown quantization backend `{backend}`")

        config = config.get_text_config(decoder=True)
        layer_types, _ = get_layer_types_and_kwargs(config)
        invalid_layer_types = set(layer_types) - {"full_attention"}
        if len(invalid_layer_types) > 0:
            raise ValueError(
                "`QuantizedCache` is only supported for models with only full attention layers. We found the following invalid layer "
                f"types: {invalid_layer_types}"
            )
        layers = [
            layer_class(nbits, axis_key, axis_value, q_group_size, residual_length)
            for _ in range(config.num_hidden_layers)
        ]
        super().__init__(layers=layers)


class EncoderDecoderCache(Cache):
    """

View on GitHub (pinned to a597f97485)

Solutions

1. Use backend='quanto' or backend='hqq' (optimum-quanto or hqq must be installed respectively)
2. Normalize/validate the backend string at config load time: backend in {'quanto', 'hqq'}
3. For bitsandbytes-style quantization, note it applies to model weights (BitsAndBytesConfig), not the KV cache via this API

Example fix

# before
cache = QuantizedCache(config, backend="bnb")

# after
cache = QuantizedCache(config, backend="hqq", nbits=4, axis_key=1, axis_value=0)

SUPPORTED = {"quanto", "hqq"}
assert backend in SUPPORTED, f"backend must be one of {SUPPORTED}, got {backend!r}"
cache = QuantizedCache(config, backend=backend)

def is_supported_kv_quant_backend(backend: str) -> bool:
    return isinstance(backend, str) and backend.lower() in {"quanto", "hqq"}

try:
    cache = QuantizedCache(config, backend=backend)
except ValueError as e:
    if "Unknown quantization backend" in str(e):
        cache = QuantizedCache(config, backend="hqq")  # explicit fallback choice
    else:
        raise

When it happens

Trigger: QuantizedCache(config, backend='bitsandbytes') or similar; passing a backend string with different casing/whitespace; passing None or an empty string.

Common situations: Confusing weight-quantization backend names (bnb, gguf, awq) with KV-cache quantization backends (quanto, hqq); typos; config-driven backend names that drift from the supported set.

Related errors

• `axis_value` for `HQQ` backend has to be one of [`0`, `1`] b
• `QuantizedCache` is only supported for models with only full
• You can construct a Cache either from a list `layers` of all
• You should provide exactly one of `layers` or `layer_class_t
• Cannot call `update_conv_state` on a non-LinearAttention lay

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/da99dbf0f46c6720. Report an issue: GitHub.