hiyouga/LlamaFactory · error · RuntimeError

NpuSwiGluKernel requires torch_npu.

Error message

NpuSwiGluKernel requires torch_npu.

What it means

The npu_fused_swiglu plugin needs torch_npu to call Ascend fused operators. The module-level import error is stored in _TORCH_NPU_IMPORT_ERROR; check_deps() converts it into RuntimeError('NpuSwiGluKernel requires torch_npu.') chained to the original failure.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/ops/mlp/npu_swiglu.py:105

        "Qwen3_5MoeMLP": npu_swiglu_forward,
    },
}


@KernelPlugin("npu_fused_swiglu").register()
class NpuSwiGluKernel(BaseKernel):
    """NPU Kernel for fused SwiGLU activation."""

    @staticmethod
    def check_device() -> None:
        current = get_current_accelerator().type
        if current != DeviceType.NPU:
            raise RuntimeError(f"NpuSwiGluKernel requires NPU, current accelerator is {current}.")

    @staticmethod
    def check_deps() -> None:
        if _TORCH_NPU_IMPORT_ERROR is not None:
            raise RuntimeError("NpuSwiGluKernel requires torch_npu.") from _TORCH_NPU_IMPORT_ERROR

    @staticmethod
    def _get_patch_forward(model_type: str, module: torch.nn.Module):
        """Return the NPU forward function for a matched SwiGLU MLP module."""
        model_patches = _MODEL_TYPE_TO_PATCHES.get(model_type, {})
        patch_forward = model_patches.get(module.__class__.__name__)
        if patch_forward is None:
            return None

        config = getattr(module, "config", None)
        if getattr(config, "hidden_act", None) != "silu":
            return None

        return patch_forward

    @staticmethod
    def _apply(**kwargs) -> "HFModel":
        """Applies the NPU fused SwiGLU kernel to the model.

View on GitHub (pinned to f28afaf635)

Solutions

  1. Install matching torch_npu for your torch+CANN versions and re-run
  2. Check the chained __cause__ for the real ImportError text
  3. Source CANN set_env.sh in the launch script/container entrypoint

Example fix

# before
# torch_npu absent → RuntimeError at check_deps
# after
pip install torch_npu==<matching-version>
Defensive patterns

Strategy: validation

Validate before calling

try:
    import torch_npu  # noqa: F401
except ImportError:
    kernels = [k for k in kernels if k != "npu_fused_swiglu"]

Prevention

When it happens

Trigger: Applying npu_fused_swiglu where `import torch_npu` fails: package absent, torch/torch_npu version mismatch, or CANN runtime not initialized.

Common situations: Same class of issues as other NPU kernels: Ascend images without the right wheel, torch upgraded independently, CANN env vars missing.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/d0e23c7c9591ebe9. Report an issue: GitHub.