hiyouga/LlamaFactory · error · RuntimeError

NpuFusedMoEKernel requires torch_npu.

Error message

NpuFusedMoEKernel requires torch_npu.

What it means

The npu_fused_moe plugin depends on torch_npu (Huawei Ascend's PyTorch adapter). The plugin captures the torch_npu import error at module load; check_deps() raises RuntimeError chained to that ImportError when torch_npu cannot be imported.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/ops/mlp/npu_fused_moe.py:386

_MODEL_TYPE_TO_PATCHES = (
    _V5_MODEL_TYPE_TO_PATCHES if is_transformers_version_greater_than("5.0.0") else _V4_MODEL_TYPE_TO_PATCHES
)


@KernelPlugin("npu_fused_moe").register()
class NpuFusedMoEKernel(BaseKernel):
    """NPU Fused MoE Kernel implementation."""

    @staticmethod
    def check_device() -> None:
        current = get_current_accelerator().type
        if current != DeviceType.NPU:
            raise RuntimeError(f"NpuFusedMoEKernel requires NPU, current accelerator is {current}.")

    @staticmethod
    def check_deps() -> None:
        if _TORCH_NPU_IMPORT_ERROR is not None:
            raise RuntimeError("NpuFusedMoEKernel requires torch_npu.") from _TORCH_NPU_IMPORT_ERROR

    @staticmethod
    def _get_patch_forward(model_type: str, module: torch.nn.Module):
        """Return the version-specific NPU forward function for a matched MoE module."""
        model_patches = _MODEL_TYPE_TO_PATCHES.get(model_type, {})
        return model_patches.get(module.__class__.__name__)

    @staticmethod
    def _apply(**kwargs) -> HFModel:
        """Applies the NPU fused MoE kernel to the model.

        Args:
            **kwargs: Keyword arguments containing the model.

        Returns:
            HFModel: The model with patched MoE forward functions.
        """
        model = kwargs["model"]

View on GitHub (pinned to f28afaf635)

Solutions

1. Install the torch_npu wheel matching your exact torch and CANN versions
2. Source the CANN environment (e.g. /usr/local/Ascend/ascend-toolkit/set_env.sh) before launching
3. Inspect __cause__ of the RuntimeError for the underlying ImportError
4. Pin torch to a version for which a compatible torch_npu wheel exists

Example fix

# before: torch_npu missing / CANN not sourced
# after
pip install torch_npu==<version-matching-torch>
source /usr/local/Ascend/ascend-toolkit/set_env.sh

try:
    import torch_npu  # noqa: F401
    npu_ok = True
except ImportError:
    npu_ok = False
if not npu_ok:
    kernels = [k for k in kernels if k != "npu_fused_moe"]

When it happens

Trigger: Applying npu_fused_moe in an environment where `import torch_npu` fails — package not installed, installed for a different torch/CANN version, or CANN toolkit environment scripts not sourced.

Common situations: Ascend Docker images missing torch_npu; torch upgraded without rebuilding torch_npu (it pins exact torch versions); CANN env (set_env.sh) not sourced so the native extension fails to load; x86 vs aarch64 wheel mismatch.

Related errors

• NpuSwiGluKernel requires torch_npu.
• NpuRMSNormKernel requires torch_npu.
• NpuRoPEKernel requires torch_npu.
• cuda_fused_moe requires Triton.
• NpuFusedMoEKernel requires NPU, current accelerator is {curr

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/18fbc1e22b31e58e. Report an issue: GitHub.