hiyouga/LlamaFactory · error · RuntimeError

cuda_fused_moe requires Triton.

Error message

cuda_fused_moe requires Triton.

What it means

cuda_fused_moe's fused Triton pipeline requires the Triton package. At import time the plugin captures any ImportError into _TRITON_IMPORT_ERROR; check_deps() re-raises it as RuntimeError('cuda_fused_moe requires Triton.') chained to the original import failure when the plugin is applied.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/ops/mlp/cuda_fused_moe.py:380

    """Pure-Triton fused MoE kernel for NVIDIA CUDA GPUs.

    Replaces HuggingFace per-expert Python loops with a fully fused Triton pipeline:
    - Forward: scatter + grouped GEMMs + gather (single kernel per GEMM)
    - Backward: all dX and dW via grouped GEMMs (no Python loops)

    Requires: CUDA GPU + Triton
    """

    @staticmethod
    def check_device() -> None:
        current = get_current_accelerator().type
        if current != DeviceType.CUDA:
            raise RuntimeError(f"CudaFusedMoEKernel requires CUDA, current accelerator is {current}.")

    @staticmethod
    def check_deps() -> None:
        if _TRITON_IMPORT_ERROR is not None:
            raise RuntimeError("cuda_fused_moe requires Triton.") from _TRITON_IMPORT_ERROR

    @staticmethod
    def _apply(**kwargs) -> HFModel:
        model = kwargs.get("model")

        archs = getattr(model.config, "architectures", None) or []
        target_mapping = None
        for arch in archs:
            if arch in _TRITON_MOE_MAPPING:
                target_mapping = _TRITON_MOE_MAPPING[arch]
                break

        if target_mapping is None:
            logger.info(
                f"cuda_fused_moe: Model architecture {archs} not supported. "
                f"Supported: {list(_TRITON_MOE_MAPPING.keys())}"
            )
            return model

View on GitHub (pinned to f28afaf635)

Solutions

1. pip install triton (or `uv add triton`) in the training environment
2. Inspect `__cause__` of the RuntimeError to see the original ImportError — it usually names the missing/broken module
3. Match triton version to your torch version (check torch/triton compatibility matrix)
4. Verify with `python -c "import triton"` in the same env/interpreter the trainer uses

Example fix

# before: ModuleNotFoundError hidden in __cause__
# after
pip install -U triton
python -c "import triton; print(triton.__version__)"

try:
    import triton  # noqa: F401
    triton_ok = True
except ImportError:
    triton_ok = False
if not triton_ok:
    kernels = [k for k in kernels if k != "cuda_fused_moe"]

When it happens

Trigger: Applying the cuda_fused_moe kernel on a CUDA machine where `import triton` failed — Triton not installed, wrong-wheel architecture, or a Triton version incompatible with the installed torch so the import raises.

Common situations: Minimal/CI images that install torch without triton; pip resolving an old or CPU-only triton wheel; upgrading torch past the triton version's compatibility range; CUDA container images missing triton.

Related errors

• CudaFusedMoEKernel requires CUDA, current accelerator is {cu
• NpuFusedMoEKernel requires torch_npu.
• `expert_model_parallel_size` must be >= 1.
• `moe_token_dispatcher_type` must be 'allgather', 'alltoall',
• The installed Transformers-KT does not provide `TrainingArgu

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/1089a6b79878205d. Report an issue: GitHub.