{"record":{"id":"68b46a654995f6a0","repo":"hiyouga/LlamaFactory","slug":"cudafusedmoekernel-requires-cuda-current-accelera","errorCode":null,"errorMessage":"CudaFusedMoEKernel requires CUDA, current accelerator is {current}.","messagePattern":"CudaFusedMoEKernel requires CUDA, current accelerator is (.+?)\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/model_plugins/kernels/ops/mlp/cuda_fused_moe.py","lineNumber":375,"sourceCode":"# ---------------------------------------------------------------------------\n\n\n@KernelPlugin(\"cuda_fused_moe\").register()\nclass CudaFusedMoEKernel(BaseKernel):\n    \"\"\"Pure-Triton fused MoE kernel for NVIDIA CUDA GPUs.\n\n    Replaces HuggingFace per-expert Python loops with a fully fused Triton pipeline:\n    - Forward: scatter + grouped GEMMs + gather (single kernel per GEMM)\n    - Backward: all dX and dW via grouped GEMMs (no Python loops)\n\n    Requires: CUDA GPU + Triton\n    \"\"\"\n\n    @staticmethod\n    def check_device() -> None:\n        current = get_current_accelerator().type\n        if current != DeviceType.CUDA:\n            raise RuntimeError(f\"CudaFusedMoEKernel requires CUDA, current accelerator is {current}.\")\n\n    @staticmethod\n    def check_deps() -> None:\n        if _TRITON_IMPORT_ERROR is not None:\n            raise RuntimeError(\"cuda_fused_moe requires Triton.\") from _TRITON_IMPORT_ERROR\n\n    @staticmethod\n    def _apply(**kwargs) -> HFModel:\n        model = kwargs.get(\"model\")\n\n        archs = getattr(model.config, \"architectures\", None) or []\n        target_mapping = None\n        for arch in archs:\n            if arch in _TRITON_MOE_MAPPING:\n                target_mapping = _TRITON_MOE_MAPPING[arch]\n                break\n\n        if target_mapping is None:","sourceCodeStart":357,"sourceCodeEnd":393,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/model_plugins/kernels/ops/mlp/cuda_fused_moe.py#L357-L393","documentation":"The cuda_fused_moe kernel plugin replaces HuggingFace per-expert MoE loops with fused Triton kernels, which only exist for NVIDIA GPUs. Before applying, check_device() compares the current accelerator type to DeviceType.CUDA and raises RuntimeError if it differs (e.g. cpu, mlu, npu, xpu).","triggerScenarios":"Registering/applying the cuda_fused_moe kernel plugin on a machine whose accelerator is not CUDA — CPU-only box, Apple silicon, or an NPU/MLU device — or before the accelerator was initialized so the type resolves to something unexpected.","commonSituations":"Sharing a config with kernels: [cuda_fused_moe] across heterogeneous clusters; running a smoke test or data-prep job on CPU with the full kernel list enabled; running on Ascend/Cambricon hardware with a CUDA-targeted config.","solutions":["Remove cuda_fused_moe from the kernels list on non-CUDA hardware","Or guard the kernel list per node, selecting npu_fused_moe on Ascend and cuda_fused_moe on NVIDIA","Run on a CUDA GPU if the fused MoE path is required","If you expected CUDA, verify torch detects the GPU (nvidia-smi, torch.cuda.is_available) and that get_current_accelerator().type reports cuda"],"exampleFix":"# before\nkernels: [cuda_fused_moe]  # fails on CPU/NPU nodes\n\n# after\nkernels: {cuda: [cuda_fused_moe], npu: [npu_fused_moe]}  # select per device","handlingStrategy":"validation","validationCode":"from llamafactory.accelerator import get_current_accelerator\nif get_current_accelerator().type != \"cuda\":\n    kernels = [k for k in kernels if k != \"cuda_fused_moe\"]","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Make kernel lists device-conditional in cluster configs","Preflight-check accelerator type before applying any device-specific kernel"],"tags":["device-check","cuda","moe","kernel-plugin"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}