hiyouga/LlamaFactory · error · RuntimeError

NpuFusedMoEKernel requires NPU, current accelerator is {curr

Error message

NpuFusedMoEKernel requires NPU, current accelerator is {current}.

What it means

The npu_fused_moe kernel plugin provides an Ascend NPU fused MoE implementation. check_device() verifies the current accelerator type is DeviceType.NPU and raises RuntimeError otherwise, preventing a NPU-specific patch from being applied to CUDA/CPU models.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/ops/mlp/npu_fused_moe.py:381

    "qwen3_5_moe": {
        "Qwen3_5MoeExperts": NpuMoeFusedV5.experts_forward,
    },
}

_MODEL_TYPE_TO_PATCHES = (
    _V5_MODEL_TYPE_TO_PATCHES if is_transformers_version_greater_than("5.0.0") else _V4_MODEL_TYPE_TO_PATCHES
)


@KernelPlugin("npu_fused_moe").register()
class NpuFusedMoEKernel(BaseKernel):
    """NPU Fused MoE Kernel implementation."""

    @staticmethod
    def check_device() -> None:
        current = get_current_accelerator().type
        if current != DeviceType.NPU:
            raise RuntimeError(f"NpuFusedMoEKernel requires NPU, current accelerator is {current}.")

    @staticmethod
    def check_deps() -> None:
        if _TORCH_NPU_IMPORT_ERROR is not None:
            raise RuntimeError("NpuFusedMoEKernel requires torch_npu.") from _TORCH_NPU_IMPORT_ERROR

    @staticmethod
    def _get_patch_forward(model_type: str, module: torch.nn.Module):
        """Return the version-specific NPU forward function for a matched MoE module."""
        model_patches = _MODEL_TYPE_TO_PATCHES.get(model_type, {})
        return model_patches.get(module.__class__.__name__)

    @staticmethod
    def _apply(**kwargs) -> HFModel:
        """Applies the NPU fused MoE kernel to the model.

        Args:
            **kwargs: Keyword arguments containing the model.

View on GitHub (pinned to f28afaf635)

Solutions

Remove npu_fused_moe from the kernels list when not running on Ascend NPU
If on Ascend hardware, ensure torch_npu is installed and initialized so the accelerator type resolves to npu
Select the fused MoE kernel per device (npu_fused_moe on NPU, cuda_fused_moe on CUDA)

Example fix

# before (running on CUDA node)
kernels: [npu_fused_moe]

# after
kernels: [cuda_fused_moe]  # or omit on this node

Defensive patterns

Strategy: validation

Validate before calling

if get_current_accelerator().type != "npu":
    kernels = [k for k in kernels if k != "npu_fused_moe"]

Prevention

Split kernel configs per hardware target
On Ascend, confirm accelerator reports npu before applying NPU kernels

When it happens

Trigger: Applying the npu_fused_moe kernel plugin when the accelerator is cuda, cpu, or any non-NPU type — e.g. a config written for Ascend hardware run on an NVIDIA machine, or vice versa config reuse.

Common situations: Heterogeneous clusters mixing NVIDIA and Ascend nodes with a single shared kernel list; local debugging on CPU/GPU of configs authored for NPU; torch_npu not activated so the accelerator resolves to cpu even on Ascend hardware.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/f0cd1b0cd2663ba0. Report an issue: GitHub.