sgl-project/sglang · critical · ImportError

aiter is required when SGLANG_USE_AITER is set to True

Error message

aiter is required when SGLANG_USE_AITER is set to True

What it means

Raised at import time of the Triton fused MoE utilities when SGLANG_USE_AITER is truthy but the AMD-oriented aiter package cannot be imported. SGLang uses aiter's moe_sum kernel as the reduction path on ROCm/HIP when explicitly requested via env var. Missing aiter makes the module import fail entirely.

Source

Thrown at python/sglang/srt/layers/moe/moe_runner/triton_utils/fused_moe.py:72

_use_aiter = get_bool_env_var("SGLANG_USE_AITER") and _is_hip
_is_xpu = is_xpu()
_is_musa = is_musa()


if _is_cuda:
    from sgl_kernel import moe_sum_reduce

    from sglang.kernels.ops.activation.activation import gelu_and_mul, silu_and_mul
elif _is_cpu and _is_cpu_amx_available:
    pass
elif _is_hip:
    from sgl_kernel import gelu_and_mul, silu_and_mul

    if _use_aiter:
        try:
            from aiter import moe_sum
        except ImportError:
            raise ImportError("aiter is required when SGLANG_USE_AITER is set to True")
    # Note: vllm_ops is not needed for HIP when _use_aiter=False
    # because the code uses moe_sum_reduce_triton as fallback (line 619)
elif _is_xpu:
    from sgl_kernel import moe_sum_reduce, silu_and_mul
elif _is_musa:
    from sgl_kernel import moe_sum_reduce

    _silu_and_mul_musa = torch.nn.SwishGLU()

# Try to import vllm_ops for non-CUDA/HIP/XPU platforms
_has_vllm_ops = False
if not _is_cuda and not _is_hip and not _is_xpu:
    try:
        from vllm import _custom_ops as vllm_ops

        _has_vllm_ops = True
    except ImportError:
        # Fallback: vllm not available, will use native PyTorch implementations

View on GitHub (pinned to 0132848349)

Solutions

1. Install or repair aiter (pip install aiter / build from https://github.com/ROCm/aiter matching your ROCm and torch version) and verify `python -c "from aiter import moe_sum"`
2. Unset SGLANG_USE_AITER (or set to 0/false) if you don't need aiter — the code falls back to moe_sum_reduce_triton on HIP
3. Verify you are actually on an AMD/ROCm platform; on NVIDIA GPUs aiter is not applicable

Example fix

# before
SGLANG_USE_AITER=1 python -m sglang.launch_server ...
# ImportError: aiter is required when SGLANG_USE_AITER is set to True

# after
pip install aiter  # ROCm-matched build
python -c "from aiter import moe_sum"  # sanity check
# or: unset SGLANG_USE_AITER

import os, importlib.util
if os.environ.get("SGLANG_USE_AITER", "").lower() in ("1","true","yes"):
    assert importlib.util.find_spec("aiter") is not None, \
        "SGLANG_USE_AITER set but aiter not installed; unset it or pip install aiter"

When it happens

Trigger: Setting SGLANG_USE_AITER=1 (or True) on a ROCm/HIP build while the aiter package is not installed or not importable in the environment, then importing sglang.srt.layers.moe (any model load triggers it).

Common situations: Copying ROCm-tuned launch configs or benchmark scripts that set SGLANG_USE_AITER without installing aiter; aiter installed for a different ROCm/torch version so the import fails; running on CUDA machines with the env var left over from a previous shell.

Related errors

• AITER Sage attention is not available, please update AITER v
• Online MXFP4 quantization for MoE layers requires an AMD ROC
• native MXFP8 MoE only supports gated swiglu-oai, got {activa
• AITer backend does not have a metadata builder.
• AITer backend requires num_heads ({num_heads}) to be a multi

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8da4be03f565c04b. Report an issue: GitHub.