{"record":{"id":"0febf074905a9f87","repo":"hiyouga/LlamaFactory","slug":"unsupported-flash-linear-attention-kernels-sorte","errorCode":null,"errorMessage":"Unsupported Flash Linear Attention kernels: {sorted(unsupported)}","messagePattern":"Unsupported Flash Linear Attention kernels: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/model_plugins/kernels/ops/linear_attention/fla.py","lineNumber":79,"sourceCode":"    def _apply(**kwargs) -> HFModel:\n        model = kwargs[\"model\"]\n        config = kwargs.get(\"config\") or {}\n        include_kernels = config.get(\"include_kernels\", \"auto\")\n        chunk_size = config.get(\"chunk_size\", 64)\n\n        if include_kernels == \"auto\" or include_kernels is True:\n            selected = list(FLASH_LINEAR_ATTENTION_KERNELS)\n        elif isinstance(include_kernels, str):\n            selected = [name.strip() for name in include_kernels.split(\",\") if name.strip()]\n        else:\n            raise TypeError(\"kernel_config.include_kernels must be 'auto' or a comma-separated string.\")\n\n        if not selected:\n            raise ValueError(\"kernel_config.include_kernels must select at least one FLA kernel.\")\n\n        unsupported = set(selected).difference(FLASH_LINEAR_ATTENTION_KERNELS)\n        if unsupported:\n            raise ValueError(f\"Unsupported Flash Linear Attention kernels: {sorted(unsupported)}\")\n        if isinstance(chunk_size, bool) or not isinstance(chunk_size, int) or chunk_size not in SUPPORTED_CHUNK_SIZES:\n            raise ValueError(f\"chunk_size must be one of {SUPPORTED_CHUNK_SIZES}, got {chunk_size!r}.\")\n\n        from fsdp_turbo.ops.registry import get_op\n        from fsdp_turbo.utils.patch import patch_model_members\n\n        patched = 0\n        named_modules = tuple(model.named_modules())\n        for op_name in selected:\n            module_attribute = FLA_MODULE_ATTRIBUTES[op_name]\n            op = get_op(op_name)\n            configured_op = partial(op, chunk_size=chunk_size) if op_name == CHUNK_GATED_DELTA_RULE else op\n            targets = {\n                f\"{type(module).__module__}.{type(module).__name__}.{module_attribute}\"\n                for _, module in named_modules\n                if callable(getattr(module, module_attribute, None))\n            }\n            matched = patch_model_members(model, sorted(targets), configured_op) if targets else 0","sourceCodeStart":61,"sourceCodeEnd":97,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/model_plugins/kernels/ops/linear_attention/fla.py#L61-L97","documentation":"The FLA plugin restricts kernel selection to the known set FLASH_LINEAR_ATTENTION_KERNELS. If any requested name is not in that registry the plugin raises a ValueError listing the unsupported names, rather than partially applying an unknown op.","triggerScenarios":"Passing include_kernels containing kernel names that are not in FLASH_LINEAR_ATTENTION_KERNELS — typos, renamed kernels from a different fsdp_turbo/FLA version, or kernels from an unrelated plugin.","commonSituations":"Version skew: kernel names changed between fsdp_turbo/fla releases but the config kept old names; copy-paste from docs for a different version; typos like 'chunked_delta_rule' vs 'chunk_gated_delta_rule'.","solutions":["Read the error message — it lists the exact unsupported names; fix typos","Check FLASH_LINEAR_ATTENTION_KERNELS in your installed fla.py to see the accepted names for your version","Set include_kernels to \"auto\" to select all supported kernels without naming them","Pin or upgrade fsdp_turbo/fla so kernel names match your config"],"exampleFix":"# before\nkernel_config:\n  include_kernels: \"chunked_gated_delta_rule\"\n\n# after\nkernel_config:\n  include_kernels: \"chunk_gated_delta_rule\"","handlingStrategy":"validation","validationCode":"from llamafactory.v1.plugins.model_plugins.kernels.ops.linear_attention.fla import FLASH_LINEAR_ATTENTION_KERNELS\nbad = set(selected).difference(FLASH_LINEAR_ATTENTION_KERNELS)\nif bad:\n    raise ValueError(f\"unknown FLA kernels: {sorted(bad)}; valid: {sorted(FLASH_LINEAR_ATTENTION_KERNELS)}\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Import the kernel registry and diff your config against it in a preflight check","Pin fsdp_turbo/fla versions in CI so kernel names can't drift"],"tags":["configuration","validation","version-skew","kernel-plugin"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}