hiyouga/LlamaFactory · error · RuntimeError

NpuRoPEKernel requires NPU, current accelerator is {current}

Error message

NpuRoPEKernel requires NPU, current accelerator is {current}.

What it means

The npu_fused_rope plugin replaces rotary-embedding forwards with an NPU implementation. check_device() requires accelerator type DeviceType.NPU and raises RuntimeError otherwise, since npu RoPE ops do not exist on other backends.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/ops/rope/npu_rope.py:136

    "qwen3_next": _default_rope_patch("qwen3_next"),
    "qwen3_omni_moe": _default_rope_patch("qwen3_omni_moe"),
    "qwen3_omni_moe_thinker": _default_rope_patch("qwen3_omni_moe"),
    "qwen3_vl": _default_rope_patch("qwen3_vl"),
    "qwen3_vl_moe": _default_rope_patch("qwen3_vl_moe"),
    "qwen3_5": _default_rope_patch("qwen3_5"),
    "qwen3_5_moe": _default_rope_patch("qwen3_5_moe"),
}


@KernelPlugin("npu_fused_rope").register()
class NpuRoPEKernel(BaseKernel):
    """NPU Kernel for Rotary Position Embedding."""

    @staticmethod
    def check_device() -> None:
        current = get_current_accelerator().type
        if current != DeviceType.NPU:
            raise RuntimeError(f"NpuRoPEKernel requires NPU, current accelerator is {current}.")

    @staticmethod
    def check_deps() -> None:
        if _TORCH_NPU_IMPORT_ERROR is not None:
            raise RuntimeError("NpuRoPEKernel requires torch_npu.") from _TORCH_NPU_IMPORT_ERROR

    @staticmethod
    def _apply_model_patches(model_type: str) -> int:
        patches = _MODEL_TYPE_TO_PATCHES.get(model_type)
        if patches is None:
            return 0

        patched_count = 0
        for module_name, replacements in patches:
            try:
                target_module = importlib.import_module(module_name)
            except Exception as e:
                logger.warning_rank0_once(f"Failed to import {module_name} for NPU RoPE kernel: {e}")

View on GitHub (pinned to f28afaf635)

Solutions

Drop npu_fused_rope from kernels on non-NPU nodes
On NPU nodes verify accelerator type resolves to npu before the run
Make kernels selection device-aware

Example fix

# before
kernels: [npu_fused_rope]  # on CUDA

# after
kernels: [flash_attn]  # device-appropriate choice

Defensive patterns

Strategy: validation

Validate before calling

if get_current_accelerator().type != "npu":
    kernels = [k for k in kernels if k != "npu_fused_rope"]

Prevention

Per-device kernel lists in heterogeneous clusters
Assert accelerator type in a startup sanity check

When it happens

Trigger: Applying npu_fused_rope on CUDA/CPU machines — wrong-hardware kernel list, or NPU hardware where the accelerator type is misreported because torch_npu is uninitialized.

Common situations: Heterogeneous cluster configs; smoke tests on laptops/GPU boxes; accelerator type queried before device backend init.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/ae4ac8fb6e1bb40f. Report an issue: GitHub.