hiyouga/LlamaFactory · error · RuntimeError

FlashLinearAttentionKernel requires CUDA or NPU, current acc

Error message

FlashLinearAttentionKernel requires CUDA or NPU, current accelerator is {current}.

What it means

FlashLinearAttentionKernel.check_device() restricts the FLA kernel to CUDA or NPU, mirroring the Liger gate. On CPU/XPU/MPS or a misdetected accelerator it raises RuntimeError with the current device type.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/ops/linear_attention/fla.py:48

    CHUNK_GATED_DELTA_RULE,
    FUSED_RECURRENT_GATED_DELTA_RULE,
)
FLA_MODULE_ATTRIBUTES = {
    CHUNK_GATED_DELTA_RULE: "chunk_gated_delta_rule",
    FUSED_RECURRENT_GATED_DELTA_RULE: "recurrent_gated_delta_rule",
}
SUPPORTED_CHUNK_SIZES = (16, 32, 64)


@KernelPlugin("flash-linear-attention").register()
class FlashLinearAttentionKernel(BaseKernel):
    """Install selected FLA callables through FSDPTurbo's device operator registry."""

    @staticmethod
    def check_device() -> None:
        current = get_current_accelerator().type
        if current not in (DeviceType.CUDA, DeviceType.NPU):
            raise RuntimeError(f"FlashLinearAttentionKernel requires CUDA or NPU, current accelerator is {current}.")

    @staticmethod
    def check_deps() -> None:
        try:
            import fla.ops.gated_delta_rule  # noqa: F401
            import fsdp_turbo.ops.fla  # noqa: F401
            from fsdp_turbo.ops.registry import get_op  # noqa: F401
            from fsdp_turbo.utils.patch import patch_model_members  # noqa: F401
        except ImportError as exc:
            raise RuntimeError("Flash Linear Attention and FSDPTurbo are required for this kernel.") from exc

    @staticmethod
    def _apply(**kwargs) -> HFModel:
        model = kwargs["model"]
        config = kwargs.get("config") or {}
        include_kernels = config.get("include_kernels", "auto")
        chunk_size = config.get("chunk_size", 64)

View on GitHub (pinned to f28afaf635)

Solutions

1. Run on a CUDA/NPU node for FLA kernels.
2. Drop 'flash-linear-attention' from kernel_config.name on other devices, or use "auto" selection which filters by device_type.
3. On NPU, verify torch_npu is imported and the accelerator registers as NPU before training.

Example fix

# before
name: "flash-linear-attention"  # on CPU runner

# after
name: "auto"  # device-appropriate kernels only

from llamafactory.v1.core.accelerator import get_current_accelerator
t = get_current_accelerator().type
assert t in ('cuda', 'npu'), f'FLA kernel needs CUDA/NPU, got {t}'

def fla_supported() -> bool:
    """True on CUDA or NPU accelerators."""
    return get_current_accelerator().type in (DeviceType.CUDA, DeviceType.NPU)

try:
    model = KernelPlugin('flash-linear-attention').apply(model=model)
except RuntimeError as e:
    if 'requires CUDA or NPU' in str(e):
        logger.warning('skipping FLA kernel on %s', get_current_accelerator().type)
    else:
        raise

When it happens

Trigger: kernel_config.name includes 'flash-linear-attention' while the detected accelerator type is not CUDA/NPU — e.g. CPU dev runs, MPS, or NPU setups where torch_npu failed to init so the type fell back to CPU.

Common situations: Shared training YAMLs run across heterogeneous nodes; linear-attention (gated delta rule) models configured globally; NPU images missing proper accelerator initialization.

Related errors

• LigerKernel requires CUDA or NPU, current accelerator is {cu
• Flash Linear Attention and FSDPTurbo are required for this k
• This device does not support `pure_bf16`.
• HFModel instance is required for {cls.__name__}.
• kernel_config.name must be a string.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/be3464a0c601fd9f. Report an issue: GitHub.