hiyouga/LlamaFactory · error · RuntimeError

NpuRoPEKernel requires torch_npu.

Error message

NpuRoPEKernel requires torch_npu.

What it means

The npu_fused_rope plugin needs torch_npu to call NPU rotary-position operators. The import error captured at load time is re-raised by check_deps() as RuntimeError chained to the original ImportError.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/ops/rope/npu_rope.py:141

    "qwen3_5": _default_rope_patch("qwen3_5"),
    "qwen3_5_moe": _default_rope_patch("qwen3_5_moe"),
}


@KernelPlugin("npu_fused_rope").register()
class NpuRoPEKernel(BaseKernel):
    """NPU Kernel for Rotary Position Embedding."""

    @staticmethod
    def check_device() -> None:
        current = get_current_accelerator().type
        if current != DeviceType.NPU:
            raise RuntimeError(f"NpuRoPEKernel requires NPU, current accelerator is {current}.")

    @staticmethod
    def check_deps() -> None:
        if _TORCH_NPU_IMPORT_ERROR is not None:
            raise RuntimeError("NpuRoPEKernel requires torch_npu.") from _TORCH_NPU_IMPORT_ERROR

    @staticmethod
    def _apply_model_patches(model_type: str) -> int:
        patches = _MODEL_TYPE_TO_PATCHES.get(model_type)
        if patches is None:
            return 0

        patched_count = 0
        for module_name, replacements in patches:
            try:
                target_module = importlib.import_module(module_name)
            except Exception as e:
                logger.warning_rank0_once(f"Failed to import {module_name} for NPU RoPE kernel: {e}")
                continue

            for target_function_name, replacement in replacements:
                if not hasattr(target_module, target_function_name):
                    logger.warning_rank0_once(f"{module_name} has no {target_function_name}, skip NPU RoPE patch.")

View on GitHub (pinned to f28afaf635)

Solutions

1. Install/reinstall torch_npu matching torch+CANN; test `python -c "import torch_npu"`
2. Inspect the chained ImportError for the exact missing symbol or library
3. Re-source the CANN environment and confirm LD_LIBRARY_PATH includes the toolkit libs

Example fix

# before
# RuntimeError: NpuRoPEKernel requires torch_npu.
# after
pip install torch_npu==<matching> ; python -c "import torch_npu"

try:
    import torch_npu  # noqa: F401
except ImportError:
    kernels = [k for k in kernels if k != "npu_fused_rope"]

When it happens

Trigger: Applying npu_fused_rope where torch_npu cannot be imported — not installed, incompatible with the installed torch, or CANN runtime environment not loaded.

Common situations: Ascend images with broken torch_npu installs; environment activated in a different shell/venv than the trainer; CANN upgrade leaving stale .so files.

Related errors

• NpuFusedMoEKernel requires torch_npu.
• NpuSwiGluKernel requires torch_npu.
• NpuRMSNormKernel requires torch_npu.
• NpuRoPEKernel requires NPU, current accelerator is {current}
• The installed Transformers-KT does not provide `TrainingArgu

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/39c0cccd4b4c72d9. Report an issue: GitHub.