huggingface/transformers · error · ValueError

Unsupported attention implementation: '{self.attn_implementa

Error message

Unsupported attention implementation: '{self.attn_implementation}'. Must be one of {VALID_ATTN_IMPLEMENTATIONS} or a kernels-community kernel (e.g. 'kernels-community/flash-attn2').

What it means

_validate_args checks --attn-implementation against a fixed allow-list {eager, sdpa, flash_attention_2, flash_attention_3, flex_attention} plus anything prefixed kernels-community/. Any other value raises ValueError at startup. Note the kernels-community escape hatch lets kernel hub ids like 'kernels-community/flash-attn2' pass.

Source

Thrown at src/transformers/cli/serving/model_manager.py:201

            )
            return "kernels-community/metal-flash-sdpa@223ca3350d7ba32ecf19341ff2cbb8c43fa47d62"
        return attn_implementation

    def _validate_args(self):
        if self.quantization is not None and self.quantization not in ("bnb-4bit", "bnb-8bit"):
            raise ValueError(
                f"Unsupported quantization method: '{self.quantization}'. Must be 'bnb-4bit' or 'bnb-8bit'."
            )
        VALID_ATTN_IMPLEMENTATIONS = {"eager", "sdpa", "flash_attention_2", "flash_attention_3", "flex_attention"}
        is_kernels_community = self.attn_implementation is not None and self.attn_implementation.startswith(
            "kernels-community/"
        )
        if (
            self.attn_implementation is not None
            and not is_kernels_community
            and self.attn_implementation not in VALID_ATTN_IMPLEMENTATIONS
        ):
            raise ValueError(
                f"Unsupported attention implementation: '{self.attn_implementation}'. "
                f"Must be one of {VALID_ATTN_IMPLEMENTATIONS} or a kernels-community kernel (e.g. 'kernels-community/flash-attn2')."
            )

    @staticmethod
    def process_model_name(model_id: str) -> str:
        """Canonicalize to `'model_id@revision'` format. Defaults to `@main`."""
        if "@" in model_id:
            return model_id
        return f"{model_id}@main"

    def get_quantization_config(self) -> BitsAndBytesConfig | None:
        """Return a BitsAndBytesConfig based on the `quantization` setting, or None."""
        if self.quantization == "bnb-4bit":
            return BitsAndBytesConfig(
                load_in_4bit=True,
                bnb_4bit_quant_type="nf4",
                bnb_4bit_use_double_quant=True,

View on GitHub (pinned to a597f97485)

Solutions

1. Use one of: eager, sdpa, flash_attention_2, flash_attention_3, flex_attention
2. For kernel-hub attention, pass the full id: --attn-implementation kernels-community/flash-attn2
3. Check exact spelling and lowercase; the comparison is exact
4. Ensure the matching backend is installed (flash-attn package for flash_attention_2/3)

Example fix

# before
transformers serve --model_id llama --attn-implementation fa2

# after
transformers serve --model_id llama --attn-implementation flash_attention_2

VALID_ATTN = {"eager", "sdpa", "flash_attention_2", "flash_attention_3", "flex_attention"}
if attn and attn not in VALID_ATTN and not attn.startswith("kernels-community/"):
    raise SystemExit(f"Unsupported attn {attn!r}")

def is_supported_attn(value: str | None) -> bool:
    if value is None:
        return True
    return value in {"eager", "sdpa", "flash_attention_2", "flash_attention_3", "flex_attention"} or value.startswith(
        "kernels-community/"
    )

When it happens

Trigger: Passing --attn-implementation flash_attention (missing the _2); 'fa2' or 'flash' shorthand; 'xformers' or other backends; a kernel id without the kernels-community/ prefix; typos like 'SDPA' (case-sensitive).

Common situations: Copy-pasting attention names from other inference servers; assuming every attention backend transformers core supports is exposed here; case/typo errors in scripts.

Related errors

• Unsupported dtype: '{dtype}'. Must be 'auto' or a valid torc
• Unsupported quantization method: '{self.quantization}'. Must
• `axis_value` for `HQQ` backend has to be one of [`0`, `1`] b
• You can construct a Cache either from a list `layers` of all
• You should provide exactly one of `layers` or `layer_class_t

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/386025e28f6f0620. Report an issue: GitHub.