hiyouga/LlamaFactory · error · RuntimeError

NpuRMSNormKernel requires NPU, current accelerator is {curre

Error message

NpuRMSNormKernel requires NPU, current accelerator is {current}.

What it means

The npu_fused_rmsnorm plugin's check_device() requires the accelerator type to be DeviceType.NPU. On CUDA/CPU/other devices it raises RuntimeError instead of patching RMSNorm modules with an NPU-only forward that would crash later.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/ops/rms_norm/npu_rms_norm.py:166

        "Qwen3_5RMSNorm": npu_residual_rms_norm_forward,
        "Qwen3_5RMSNormGated": npu_gated_rms_norm_forward,
    },
    "qwen3_5_moe": {
        "Qwen3_5MoeRMSNorm": npu_residual_rms_norm_forward,
        "Qwen3_5MoeRMSNormGated": npu_gated_rms_norm_forward,
    },
}


@KernelPlugin("npu_fused_rmsnorm").register()
class NpuRMSNormKernel(BaseKernel):
    """NPU kernel wrapper for RMSNorm that applies the replacement within a model."""

    @staticmethod
    def check_device() -> None:
        current = get_current_accelerator().type
        if current != DeviceType.NPU:
            raise RuntimeError(f"NpuRMSNormKernel requires NPU, current accelerator is {current}.")

    @staticmethod
    def check_deps() -> None:
        if _TORCH_NPU_IMPORT_ERROR is not None:
            raise RuntimeError("NpuRMSNormKernel requires torch_npu.") from _TORCH_NPU_IMPORT_ERROR

    @staticmethod
    def _get_patch_forward(model_type: str, module: torch.nn.Module):
        """Return the NPU forward function for a matched RMSNorm module."""
        model_patches = _MODEL_TYPE_TO_PATCHES.get(model_type, {})
        return model_patches.get(module.__class__.__name__)

    @staticmethod
    def _apply(**kwargs) -> "HFModel":
        """Iterate the model and apply NPU-optimized forward to matched RMSNorm modules.

        Matches modules configured for the current model type, then binds the corresponding
        NPU-optimized forward function as an instance method via ``types.MethodType`` to

View on GitHub (pinned to f28afaf635)

Solutions

Remove npu_fused_rmsnorm from kernels on non-NPU nodes
On Ascend, install/initialize torch_npu before plugin checks run
Gate the kernel list on the detected accelerator type

Example fix

# before
kernels: [npu_fused_rmsnorm]  # on CUDA

# after
kernels: []

Defensive patterns

Strategy: validation

Validate before calling

if get_current_accelerator().type != "npu":
    kernels = [k for k in kernels if k != "npu_fused_rmsnorm"]

Prevention

Gate NPU kernels on detected accelerator type
Initialize device backend before plugin application

When it happens

Trigger: Enabling npu_fused_rmsnorm on non-Ascend hardware, or on Ascend hardware where torch_npu is not initialized so get_current_accelerator().type is not npu.

Common situations: Device-agnostic configs listing all NPU kernels run on GPU nodes; accelerator queried before device setup completes.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/440d720452430d44. Report an issue: GitHub.