{"record":{"id":"18fbc1e22b31e58e","repo":"hiyouga/LlamaFactory","slug":"npufusedmoekernel-requires-torch-npu","errorCode":null,"errorMessage":"NpuFusedMoEKernel requires torch_npu.","messagePattern":"NpuFusedMoEKernel requires torch_npu\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/model_plugins/kernels/ops/mlp/npu_fused_moe.py","lineNumber":386,"sourceCode":"_MODEL_TYPE_TO_PATCHES = (\n    _V5_MODEL_TYPE_TO_PATCHES if is_transformers_version_greater_than(\"5.0.0\") else _V4_MODEL_TYPE_TO_PATCHES\n)\n\n\n@KernelPlugin(\"npu_fused_moe\").register()\nclass NpuFusedMoEKernel(BaseKernel):\n    \"\"\"NPU Fused MoE Kernel implementation.\"\"\"\n\n    @staticmethod\n    def check_device() -> None:\n        current = get_current_accelerator().type\n        if current != DeviceType.NPU:\n            raise RuntimeError(f\"NpuFusedMoEKernel requires NPU, current accelerator is {current}.\")\n\n    @staticmethod\n    def check_deps() -> None:\n        if _TORCH_NPU_IMPORT_ERROR is not None:\n            raise RuntimeError(\"NpuFusedMoEKernel requires torch_npu.\") from _TORCH_NPU_IMPORT_ERROR\n\n    @staticmethod\n    def _get_patch_forward(model_type: str, module: torch.nn.Module):\n        \"\"\"Return the version-specific NPU forward function for a matched MoE module.\"\"\"\n        model_patches = _MODEL_TYPE_TO_PATCHES.get(model_type, {})\n        return model_patches.get(module.__class__.__name__)\n\n    @staticmethod\n    def _apply(**kwargs) -> HFModel:\n        \"\"\"Applies the NPU fused MoE kernel to the model.\n\n        Args:\n            **kwargs: Keyword arguments containing the model.\n\n        Returns:\n            HFModel: The model with patched MoE forward functions.\n        \"\"\"\n        model = kwargs[\"model\"]","sourceCodeStart":368,"sourceCodeEnd":404,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/model_plugins/kernels/ops/mlp/npu_fused_moe.py#L368-L404","documentation":"The npu_fused_moe plugin depends on torch_npu (Huawei Ascend's PyTorch adapter). The plugin captures the torch_npu import error at module load; check_deps() raises RuntimeError chained to that ImportError when torch_npu cannot be imported.","triggerScenarios":"Applying npu_fused_moe in an environment where `import torch_npu` fails — package not installed, installed for a different torch/CANN version, or CANN toolkit environment scripts not sourced.","commonSituations":"Ascend Docker images missing torch_npu; torch upgraded without rebuilding torch_npu (it pins exact torch versions); CANN env (set_env.sh) not sourced so the native extension fails to load; x86 vs aarch64 wheel mismatch.","solutions":["Install the torch_npu wheel matching your exact torch and CANN versions","Source the CANN environment (e.g. /usr/local/Ascend/ascend-toolkit/set_env.sh) before launching","Inspect __cause__ of the RuntimeError for the underlying ImportError","Pin torch to a version for which a compatible torch_npu wheel exists"],"exampleFix":"# before: torch_npu missing / CANN not sourced\n# after\npip install torch_npu==<version-matching-torch>\nsource /usr/local/Ascend/ascend-toolkit/set_env.sh","handlingStrategy":"validation","validationCode":"try:\n    import torch_npu  # noqa: F401\n    npu_ok = True\nexcept ImportError:\n    npu_ok = False\nif not npu_ok:\n    kernels = [k for k in kernels if k != \"npu_fused_moe\"]","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pin torch_npu to the exact matching torch+CANN version","Source CANN set_env.sh in container entrypoints"],"tags":["dependencies","npu","torch-npu","moe"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}