hiyouga/LlamaFactory · error · TypeError

kernel_config.include_kernels must be 'auto' or a comma-sepa

Error message

kernel_config.include_kernels must be 'auto' or a comma-separated string.

What it means

The FLA (Flash Linear Attention) kernel plugin validates the `kernel_config.include_kernels` setting before patching the model. The value must be the string 'auto' (or boolean True) to select all kernels, or a non-empty comma-separated string of kernel names. Any other type (int, list, dict, None explicitly passed, nested config object) is rejected with a TypeError because the plugin cannot interpret it.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/ops/linear_attention/fla.py:72

            import fsdp_turbo.ops.fla  # noqa: F401
            from fsdp_turbo.ops.registry import get_op  # noqa: F401
            from fsdp_turbo.utils.patch import patch_model_members  # noqa: F401
        except ImportError as exc:
            raise RuntimeError("Flash Linear Attention and FSDPTurbo are required for this kernel.") from exc

    @staticmethod
    def _apply(**kwargs) -> HFModel:
        model = kwargs["model"]
        config = kwargs.get("config") or {}
        include_kernels = config.get("include_kernels", "auto")
        chunk_size = config.get("chunk_size", 64)

        if include_kernels == "auto" or include_kernels is True:
            selected = list(FLASH_LINEAR_ATTENTION_KERNELS)
        elif isinstance(include_kernels, str):
            selected = [name.strip() for name in include_kernels.split(",") if name.strip()]
        else:
            raise TypeError("kernel_config.include_kernels must be 'auto' or a comma-separated string.")

        if not selected:
            raise ValueError("kernel_config.include_kernels must select at least one FLA kernel.")

        unsupported = set(selected).difference(FLASH_LINEAR_ATTENTION_KERNELS)
        if unsupported:
            raise ValueError(f"Unsupported Flash Linear Attention kernels: {sorted(unsupported)}")
        if isinstance(chunk_size, bool) or not isinstance(chunk_size, int) or chunk_size not in SUPPORTED_CHUNK_SIZES:
            raise ValueError(f"chunk_size must be one of {SUPPORTED_CHUNK_SIZES}, got {chunk_size!r}.")

        from fsdp_turbo.ops.registry import get_op
        from fsdp_turbo.utils.patch import patch_model_members

        patched = 0
        named_modules = tuple(model.named_modules())
        for op_name in selected:
            module_attribute = FLA_MODULE_ATTRIBUTES[op_name]
            op = get_op(op_name)

View on GitHub (pinned to f28afaf635)

Solutions

1. Set include_kernels to the string "auto" to enable all FLA kernels
2. Or set it to a comma-separated string of kernel names, e.g. "chunk_gated_delta_rule,chunk_fwd"
3. Check the code path that builds kernel_config and ensure it serializes lists to strings before the plugin sees it

Example fix

# before
kernel_config:
  include_kernels:
    - chunk_gated_delta_rule
    - chunk_fwd

# after
kernel_config:
  include_kernels: "chunk_gated_delta_rule,chunk_fwd"  # or "auto"

def normalize_include_kernels(cfg: dict) -> None:
    v = cfg.get("include_kernels", "auto")
    if not (v == "auto" or v is True or isinstance(v, str)):
        raise TypeError(f"include_kernels must be 'auto' or str, got {type(v).__name__}")
    if isinstance(v, list):
        cfg["include_kernels"] = ",".join(v)

def is_valid_include_kernels(v) -> bool:
    return v is True or (isinstance(v, str) and not isinstance(v, bool))

When it happens

Trigger: Calling the FLA kernel plugin with a kernel_config dict where include_kernels is a Python list (e.g. ['chunk_gated_delta_rule']), a dict, an int, or None instead of the string 'auto' or 'kern1,kern2'. Happens when YAML/JSON config values are parsed into native structures and passed through unchanged.

Common situations: Users writing `include_kernels: [chunk_fwd, chunk_bwd]` as a YAML list instead of a comma-separated string; passing a parsed JSON array; or copying a list-style config from another kernel plugin that accepts sequences.

Related errors

• kernel_config.include_kernels must select at least one FLA k
• Unsupported Flash Linear Attention kernels: {sorted(unsuppor
• chunk_size must be one of {SUPPORTED_CHUNK_SIZES}, got {chun
• Unknown backend: {model_args.infer_backend}
• LLaMA-Factory `kt_config` must be a flat mapping.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/fafb36c4931a1519. Report an issue: GitHub.