hiyouga/LlamaFactory · error · RuntimeError

NpuSwiGluKernel requires NPU, current accelerator is {curren

Error message

NpuSwiGluKernel requires NPU, current accelerator is {current}.

What it means

The npu_fused_swiglu plugin replaces SwiGLU MLP forwards with an NPU-fused implementation. check_device() requires the accelerator type to be DeviceType.NPU and raises RuntimeError on any other device, since the replacement forward only exists for Ascend.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/ops/mlp/npu_swiglu.py:100

    },
    "qwen3_5": {
        "Qwen3_5MLP": npu_swiglu_forward,
    },
    "qwen3_5_moe": {
        "Qwen3_5MoeMLP": npu_swiglu_forward,
    },
}


@KernelPlugin("npu_fused_swiglu").register()
class NpuSwiGluKernel(BaseKernel):
    """NPU Kernel for fused SwiGLU activation."""

    @staticmethod
    def check_device() -> None:
        current = get_current_accelerator().type
        if current != DeviceType.NPU:
            raise RuntimeError(f"NpuSwiGluKernel requires NPU, current accelerator is {current}.")

    @staticmethod
    def check_deps() -> None:
        if _TORCH_NPU_IMPORT_ERROR is not None:
            raise RuntimeError("NpuSwiGluKernel requires torch_npu.") from _TORCH_NPU_IMPORT_ERROR

    @staticmethod
    def _get_patch_forward(model_type: str, module: torch.nn.Module):
        """Return the NPU forward function for a matched SwiGLU MLP module."""
        model_patches = _MODEL_TYPE_TO_PATCHES.get(model_type, {})
        patch_forward = model_patches.get(module.__class__.__name__)
        if patch_forward is None:
            return None

        config = getattr(module, "config", None)
        if getattr(config, "hidden_act", None) != "silu":
            return None

View on GitHub (pinned to f28afaf635)

Solutions

Drop npu_fused_swiglu from kernels on non-NPU nodes
On Ascend nodes, confirm `torch.npu.is_available()` / accelerator type is npu before applying
Make the kernel list device-conditional

Example fix

# before
kernels: [npu_fused_swiglu]  # on CUDA node

# after
kernels: []  # or device-appropriate kernel

Defensive patterns

Strategy: validation

Validate before calling

if get_current_accelerator().type != "npu":
    kernels = [k for k in kernels if k != "npu_fused_swiglu"]

Prevention

Device-gate NPU kernels in shared configs
Test accelerator detection early in the launch script

When it happens

Trigger: Applying the npu_fused_swiglu kernel plugin on cuda/cpu/other accelerators — wrong-hardware config, or Ascend hardware where torch_npu is not initialized so the accelerator reports cpu.

Common situations: Shared configs across CUDA and NPU clusters; debugging NPU configs on a GPU workstation; accelerator not yet initialized when the plugin check runs.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/ad175023574c138c. Report an issue: GitHub.