{"record":{"id":"f0cd1b0cd2663ba0","repo":"hiyouga/LlamaFactory","slug":"npufusedmoekernel-requires-npu-current-accelerato","errorCode":null,"errorMessage":"NpuFusedMoEKernel requires NPU, current accelerator is {current}.","messagePattern":"NpuFusedMoEKernel requires NPU, current accelerator is (.+?)\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/model_plugins/kernels/ops/mlp/npu_fused_moe.py","lineNumber":381,"sourceCode":"    \"qwen3_5_moe\": {\n        \"Qwen3_5MoeExperts\": NpuMoeFusedV5.experts_forward,\n    },\n}\n\n_MODEL_TYPE_TO_PATCHES = (\n    _V5_MODEL_TYPE_TO_PATCHES if is_transformers_version_greater_than(\"5.0.0\") else _V4_MODEL_TYPE_TO_PATCHES\n)\n\n\n@KernelPlugin(\"npu_fused_moe\").register()\nclass NpuFusedMoEKernel(BaseKernel):\n    \"\"\"NPU Fused MoE Kernel implementation.\"\"\"\n\n    @staticmethod\n    def check_device() -> None:\n        current = get_current_accelerator().type\n        if current != DeviceType.NPU:\n            raise RuntimeError(f\"NpuFusedMoEKernel requires NPU, current accelerator is {current}.\")\n\n    @staticmethod\n    def check_deps() -> None:\n        if _TORCH_NPU_IMPORT_ERROR is not None:\n            raise RuntimeError(\"NpuFusedMoEKernel requires torch_npu.\") from _TORCH_NPU_IMPORT_ERROR\n\n    @staticmethod\n    def _get_patch_forward(model_type: str, module: torch.nn.Module):\n        \"\"\"Return the version-specific NPU forward function for a matched MoE module.\"\"\"\n        model_patches = _MODEL_TYPE_TO_PATCHES.get(model_type, {})\n        return model_patches.get(module.__class__.__name__)\n\n    @staticmethod\n    def _apply(**kwargs) -> HFModel:\n        \"\"\"Applies the NPU fused MoE kernel to the model.\n\n        Args:\n            **kwargs: Keyword arguments containing the model.","sourceCodeStart":363,"sourceCodeEnd":399,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/model_plugins/kernels/ops/mlp/npu_fused_moe.py#L363-L399","documentation":"The npu_fused_moe kernel plugin provides an Ascend NPU fused MoE implementation. check_device() verifies the current accelerator type is DeviceType.NPU and raises RuntimeError otherwise, preventing a NPU-specific patch from being applied to CUDA/CPU models.","triggerScenarios":"Applying the npu_fused_moe kernel plugin when the accelerator is cuda, cpu, or any non-NPU type — e.g. a config written for Ascend hardware run on an NVIDIA machine, or vice versa config reuse.","commonSituations":"Heterogeneous clusters mixing NVIDIA and Ascend nodes with a single shared kernel list; local debugging on CPU/GPU of configs authored for NPU; torch_npu not activated so the accelerator resolves to cpu even on Ascend hardware.","solutions":["Remove npu_fused_moe from the kernels list when not running on Ascend NPU","If on Ascend hardware, ensure torch_npu is installed and initialized so the accelerator type resolves to npu","Select the fused MoE kernel per device (npu_fused_moe on NPU, cuda_fused_moe on CUDA)"],"exampleFix":"# before (running on CUDA node)\nkernels: [npu_fused_moe]\n\n# after\nkernels: [cuda_fused_moe]  # or omit on this node","handlingStrategy":"validation","validationCode":"if get_current_accelerator().type != \"npu\":\n    kernels = [k for k in kernels if k != \"npu_fused_moe\"]","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Split kernel configs per hardware target","On Ascend, confirm accelerator reports npu before applying NPU kernels"],"tags":["device-check","npu","moe","kernel-plugin"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}