{"record":{"id":"1089a6b79878205d","repo":"hiyouga/LlamaFactory","slug":"cuda-fused-moe-requires-triton","errorCode":null,"errorMessage":"cuda_fused_moe requires Triton.","messagePattern":"cuda_fused_moe requires Triton\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/model_plugins/kernels/ops/mlp/cuda_fused_moe.py","lineNumber":380,"sourceCode":"    \"\"\"Pure-Triton fused MoE kernel for NVIDIA CUDA GPUs.\n\n    Replaces HuggingFace per-expert Python loops with a fully fused Triton pipeline:\n    - Forward: scatter + grouped GEMMs + gather (single kernel per GEMM)\n    - Backward: all dX and dW via grouped GEMMs (no Python loops)\n\n    Requires: CUDA GPU + Triton\n    \"\"\"\n\n    @staticmethod\n    def check_device() -> None:\n        current = get_current_accelerator().type\n        if current != DeviceType.CUDA:\n            raise RuntimeError(f\"CudaFusedMoEKernel requires CUDA, current accelerator is {current}.\")\n\n    @staticmethod\n    def check_deps() -> None:\n        if _TRITON_IMPORT_ERROR is not None:\n            raise RuntimeError(\"cuda_fused_moe requires Triton.\") from _TRITON_IMPORT_ERROR\n\n    @staticmethod\n    def _apply(**kwargs) -> HFModel:\n        model = kwargs.get(\"model\")\n\n        archs = getattr(model.config, \"architectures\", None) or []\n        target_mapping = None\n        for arch in archs:\n            if arch in _TRITON_MOE_MAPPING:\n                target_mapping = _TRITON_MOE_MAPPING[arch]\n                break\n\n        if target_mapping is None:\n            logger.info(\n                f\"cuda_fused_moe: Model architecture {archs} not supported. \"\n                f\"Supported: {list(_TRITON_MOE_MAPPING.keys())}\"\n            )\n            return model","sourceCodeStart":362,"sourceCodeEnd":398,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/model_plugins/kernels/ops/mlp/cuda_fused_moe.py#L362-L398","documentation":"cuda_fused_moe's fused Triton pipeline requires the Triton package. At import time the plugin captures any ImportError into _TRITON_IMPORT_ERROR; check_deps() re-raises it as RuntimeError('cuda_fused_moe requires Triton.') chained to the original import failure when the plugin is applied.","triggerScenarios":"Applying the cuda_fused_moe kernel on a CUDA machine where `import triton` failed — Triton not installed, wrong-wheel architecture, or a Triton version incompatible with the installed torch so the import raises.","commonSituations":"Minimal/CI images that install torch without triton; pip resolving an old or CPU-only triton wheel; upgrading torch past the triton version's compatibility range; CUDA container images missing triton.","solutions":["pip install triton (or `uv add triton`) in the training environment","Inspect `__cause__` of the RuntimeError to see the original ImportError — it usually names the missing/broken module","Match triton version to your torch version (check torch/triton compatibility matrix)","Verify with `python -c \"import triton\"` in the same env/interpreter the trainer uses"],"exampleFix":"# before: ModuleNotFoundError hidden in __cause__\n# after\npip install -U triton\npython -c \"import triton; print(triton.__version__)\"","handlingStrategy":"validation","validationCode":"try:\n    import triton  # noqa: F401\n    triton_ok = True\nexcept ImportError:\n    triton_ok = False\nif not triton_ok:\n    kernels = [k for k in kernels if k != \"cuda_fused_moe\"]","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Verify `import triton` in the same interpreter the trainer uses","Bake triton into training Docker images"],"tags":["dependencies","triton","cuda","moe"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}