{"record":{"id":"c707ce8865e70653","repo":"hiyouga/LlamaFactory","slug":"unsupported-flash-attn-self-flash-attn-suppo","errorCode":null,"errorMessage":"Unsupported `flash_attn`: {self.flash_attn}. Supported values are: {supported_flash_attn}.","messagePattern":"Unsupported `flash_attn`: (.+?)\\. Supported values are: (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/config/model_args.py","lineNumber":66,"sourceCode":"        metadata={\"help\": \"Initialization configuration for the model.\"},\n    )\n    peft_config: PluginConfig | None = field(\n        default=None,\n        metadata={\"help\": \"PEFT configuration for the model.\"},\n    )\n    kernel_config: PluginConfig | None = field(\n        default=None,\n        metadata={\"help\": \"Kernel configuration for the model.\"},\n    )\n    quant_config: PluginConfig | None = field(\n        default=None,\n        metadata={\"help\": \"Quantization configuration for the model.\"},\n    )\n\n    def __post_init__(self) -> None:\n        supported_flash_attn = [item.value for item in AttentionFunction]\n        if self.flash_attn not in supported_flash_attn:\n            raise ValueError(\n                f\"Unsupported `flash_attn`: {self.flash_attn}. Supported values are: {supported_flash_attn}.\"\n            )\n\n        self.init_config = get_plugin_config(self.init_config)\n        self.peft_config = get_plugin_config(self.peft_config)\n        self.kernel_config = get_plugin_config(self.kernel_config)\n        self.quant_config = get_plugin_config(self.quant_config)\n","sourceCodeStart":48,"sourceCodeEnd":74,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/config/model_args.py#L48-L74","documentation":"`ModelArguments.__post_init__` in the v1 config validates `flash_attn` against the values of the `AttentionFunction` enum and raises this ValueError for anything else. The supported set comes from `AttentionFunction` members (e.g. `auto`, `sdpa`, `flash_attention_2`, `full`).","triggerScenarios":"Setting `flash_attn: fa2`, `flash_attn: flash_attention`, `flash_attn: true`, or any string not in the enum when constructing v1 `ModelArguments` or training via the v1 launcher.","commonSituations":"Copying v0-era shorthand (`fa2`) or boolean values from old configs; version drift where a renamed enum value no longer exists; typos.","solutions":["Use one of the values printed in the error's `supported_flash_attn` list, typically `flash_attention_2`","Install/verify flash-attn is available if you pick `flash_attention_2`","If unsure, use `auto` or `sdpa` which need no extra kernels"],"exampleFix":"# before (yaml)\nmodel:\n  flash_attn: fa2\n\n# after (yaml)\nmodel:\n  flash_attn: flash_attention_2","handlingStrategy":"validation","validationCode":"from llamafactory.v1.config import AttentionFunction  # enum module path\n\nSUPPORTED = {item.value for item in AttentionFunction}\n\ndef validate_flash_attn(value: str) -> str:\n    if value not in SUPPORTED:\n        raise SystemExit(f\"flash_attn must be one of {sorted(SUPPORTED)}, got {value}\")\n    return value","typeGuard":"def is_supported_flash_attn(value: object) -> bool:\n    return isinstance(value, str) and value in {i.value for i in AttentionFunction}","tryCatchPattern":null,"preventionTips":["Use exact enum strings (`flash_attention_2`, `sdpa`, `auto`) from AttentionFunction","Reject booleans in config validation — flash_attn is a string enum in v1","Add a JSON-schema/enum constraint to shared config tooling"],"tags":["v1","configuration","attention","enum-validation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}