{"record":{"id":"440d720452430d44","repo":"hiyouga/LlamaFactory","slug":"npurmsnormkernel-requires-npu-current-accelerator","errorCode":null,"errorMessage":"NpuRMSNormKernel requires NPU, current accelerator is {current}.","messagePattern":"NpuRMSNormKernel requires NPU, current accelerator is (.+?)\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/model_plugins/kernels/ops/rms_norm/npu_rms_norm.py","lineNumber":166,"sourceCode":"        \"Qwen3_5RMSNorm\": npu_residual_rms_norm_forward,\n        \"Qwen3_5RMSNormGated\": npu_gated_rms_norm_forward,\n    },\n    \"qwen3_5_moe\": {\n        \"Qwen3_5MoeRMSNorm\": npu_residual_rms_norm_forward,\n        \"Qwen3_5MoeRMSNormGated\": npu_gated_rms_norm_forward,\n    },\n}\n\n\n@KernelPlugin(\"npu_fused_rmsnorm\").register()\nclass NpuRMSNormKernel(BaseKernel):\n    \"\"\"NPU kernel wrapper for RMSNorm that applies the replacement within a model.\"\"\"\n\n    @staticmethod\n    def check_device() -> None:\n        current = get_current_accelerator().type\n        if current != DeviceType.NPU:\n            raise RuntimeError(f\"NpuRMSNormKernel requires NPU, current accelerator is {current}.\")\n\n    @staticmethod\n    def check_deps() -> None:\n        if _TORCH_NPU_IMPORT_ERROR is not None:\n            raise RuntimeError(\"NpuRMSNormKernel requires torch_npu.\") from _TORCH_NPU_IMPORT_ERROR\n\n    @staticmethod\n    def _get_patch_forward(model_type: str, module: torch.nn.Module):\n        \"\"\"Return the NPU forward function for a matched RMSNorm module.\"\"\"\n        model_patches = _MODEL_TYPE_TO_PATCHES.get(model_type, {})\n        return model_patches.get(module.__class__.__name__)\n\n    @staticmethod\n    def _apply(**kwargs) -> \"HFModel\":\n        \"\"\"Iterate the model and apply NPU-optimized forward to matched RMSNorm modules.\n\n        Matches modules configured for the current model type, then binds the corresponding\n        NPU-optimized forward function as an instance method via ``types.MethodType`` to","sourceCodeStart":148,"sourceCodeEnd":184,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/model_plugins/kernels/ops/rms_norm/npu_rms_norm.py#L148-L184","documentation":"The npu_fused_rmsnorm plugin's check_device() requires the accelerator type to be DeviceType.NPU. On CUDA/CPU/other devices it raises RuntimeError instead of patching RMSNorm modules with an NPU-only forward that would crash later.","triggerScenarios":"Enabling npu_fused_rmsnorm on non-Ascend hardware, or on Ascend hardware where torch_npu is not initialized so get_current_accelerator().type is not npu.","commonSituations":"Device-agnostic configs listing all NPU kernels run on GPU nodes; accelerator queried before device setup completes.","solutions":["Remove npu_fused_rmsnorm from kernels on non-NPU nodes","On Ascend, install/initialize torch_npu before plugin checks run","Gate the kernel list on the detected accelerator type"],"exampleFix":"# before\nkernels: [npu_fused_rmsnorm]  # on CUDA\n\n# after\nkernels: []","handlingStrategy":"validation","validationCode":"if get_current_accelerator().type != \"npu\":\n    kernels = [k for k in kernels if k != \"npu_fused_rmsnorm\"]","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Gate NPU kernels on detected accelerator type","Initialize device backend before plugin application"],"tags":["device-check","npu","rmsnorm","kernel-plugin"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}