{"record":{"id":"386025e28f6f0620","repo":"huggingface/transformers","slug":"unsupported-attention-implementation-self-attn","errorCode":null,"errorMessage":"Unsupported attention implementation: '{self.attn_implementation}'. Must be one of {VALID_ATTN_IMPLEMENTATIONS} or a kernels-community kernel (e.g. 'kernels-community/flash-attn2').","messagePattern":"Unsupported attention implementation: '(.+?)'\\. Must be one of (.+?) or a kernels-community kernel \\(e\\.g\\. 'kernels-community/flash-attn2'\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/cli/serving/model_manager.py","lineNumber":201,"sourceCode":"            )\n            return \"kernels-community/metal-flash-sdpa@223ca3350d7ba32ecf19341ff2cbb8c43fa47d62\"\n        return attn_implementation\n\n    def _validate_args(self):\n        if self.quantization is not None and self.quantization not in (\"bnb-4bit\", \"bnb-8bit\"):\n            raise ValueError(\n                f\"Unsupported quantization method: '{self.quantization}'. Must be 'bnb-4bit' or 'bnb-8bit'.\"\n            )\n        VALID_ATTN_IMPLEMENTATIONS = {\"eager\", \"sdpa\", \"flash_attention_2\", \"flash_attention_3\", \"flex_attention\"}\n        is_kernels_community = self.attn_implementation is not None and self.attn_implementation.startswith(\n            \"kernels-community/\"\n        )\n        if (\n            self.attn_implementation is not None\n            and not is_kernels_community\n            and self.attn_implementation not in VALID_ATTN_IMPLEMENTATIONS\n        ):\n            raise ValueError(\n                f\"Unsupported attention implementation: '{self.attn_implementation}'. \"\n                f\"Must be one of {VALID_ATTN_IMPLEMENTATIONS} or a kernels-community kernel (e.g. 'kernels-community/flash-attn2').\"\n            )\n\n    @staticmethod\n    def process_model_name(model_id: str) -> str:\n        \"\"\"Canonicalize to `'model_id@revision'` format. Defaults to `@main`.\"\"\"\n        if \"@\" in model_id:\n            return model_id\n        return f\"{model_id}@main\"\n\n    def get_quantization_config(self) -> BitsAndBytesConfig | None:\n        \"\"\"Return a BitsAndBytesConfig based on the `quantization` setting, or None.\"\"\"\n        if self.quantization == \"bnb-4bit\":\n            return BitsAndBytesConfig(\n                load_in_4bit=True,\n                bnb_4bit_quant_type=\"nf4\",\n                bnb_4bit_use_double_quant=True,","sourceCodeStart":183,"sourceCodeEnd":219,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cli/serving/model_manager.py#L183-L219","documentation":"_validate_args checks --attn-implementation against a fixed allow-list {eager, sdpa, flash_attention_2, flash_attention_3, flex_attention} plus anything prefixed kernels-community/. Any other value raises ValueError at startup. Note the kernels-community escape hatch lets kernel hub ids like 'kernels-community/flash-attn2' pass.","triggerScenarios":"Passing --attn-implementation flash_attention (missing the _2); 'fa2' or 'flash' shorthand; 'xformers' or other backends; a kernel id without the kernels-community/ prefix; typos like 'SDPA' (case-sensitive).","commonSituations":"Copy-pasting attention names from other inference servers; assuming every attention backend transformers core supports is exposed here; case/typo errors in scripts.","solutions":["Use one of: eager, sdpa, flash_attention_2, flash_attention_3, flex_attention","For kernel-hub attention, pass the full id: --attn-implementation kernels-community/flash-attn2","Check exact spelling and lowercase; the comparison is exact","Ensure the matching backend is installed (flash-attn package for flash_attention_2/3)"],"exampleFix":"# before\ntransformers serve --model_id llama --attn-implementation fa2\n\n# after\ntransformers serve --model_id llama --attn-implementation flash_attention_2","handlingStrategy":"validation","validationCode":"VALID_ATTN = {\"eager\", \"sdpa\", \"flash_attention_2\", \"flash_attention_3\", \"flex_attention\"}\nif attn and attn not in VALID_ATTN and not attn.startswith(\"kernels-community/\"):\n    raise SystemExit(f\"Unsupported attn {attn!r}\")","typeGuard":"def is_supported_attn(value: str | None) -> bool:\n    if value is None:\n        return True\n    return value in {\"eager\", \"sdpa\", \"flash_attention_2\", \"flash_attention_3\", \"flex_attention\"} or value.startswith(\n        \"kernels-community/\"\n    )","tryCatchPattern":null,"preventionTips":["Map shorthands (fa2->flash_attention_2) before calling serve","Copy the exact allow-list into wrapper scripts and keep it in sync","Prefix kernel-hub ids with kernels-community/ and pin the kernel hash"],"tags":["serving","attention","cli","validation","valueerror"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}