sgl-project/sglang · critical · ImportError

DeepEP is not installed. Please install DeepEP package from

Error message

DeepEP is not installed. Please install DeepEP package from https://github.com/deepseek-ai/deepep.

What it means

DeepEPTokenDispatcher requires the DeepEP package (deepseek-ai/deepep) for expert-parallel communication; constructing it when use_deepep is false — i.e. the import of DeepEP failed at module load — raises this ImportError immediately in __init__. It fires when the server/model config selects deepep (e.g. --moe-a2a-backend deepep) but the wheel isn't installed or importable.

Source

Thrown at python/sglang/srt/layers/moe/token_dispatcher/deepep.py:376

        if cls._instance is None:
            cls._instance = DeepEPConfig()
        return cls._instance


class _DeepEPDispatcherImplBase:
    def __init__(
        self,
        group: torch.distributed.ProcessGroup,
        router_topk: int,
        permute_fusion: bool,
        num_experts: int,
        num_local_experts: int,
        hidden_size: int,
        params_dtype: torch.dtype,
        deepep_mode: DeepEPMode,
    ):
        if not use_deepep:
            raise ImportError(
                "DeepEP is not installed. Please install DeepEP package from "
                "https://github.com/deepseek-ai/deepep."
            )

        self.group = group
        self.router_topk = router_topk
        self.permute_fusion = permute_fusion
        self.num_experts = num_experts
        self.num_local_experts = num_local_experts
        self.hidden_size = hidden_size
        self.params_dtype = params_dtype
        self.deepep_mode = deepep_mode

        self.params_bytes = 2
        # A large value will lead to large memory occupation, thus users should change it accordingly
        self.num_max_dispatch_tokens_per_rank = (
            envs.SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK.get()
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Install DeepEP: clone https://github.com/deepseek-ai/deepep and build/install matching your torch+CUDA (needs nvshmem and an IB/RDMA-capable or emulated environment); verify `python -c "import deep_ep"`
2. If you don't need EP all-to-all, switch --moe-a2a-backend to none/naive so DeepEPTokenDispatcher is never constructed
3. Use an SGLang docker image that bundles deepep

Example fix

# before
python -m sglang.launch_server ... --moe-a2a-backend deepep
# ImportError: DeepEP is not installed...

# after
git clone https://github.com/deepseek-ai/deepep && cd deepep && pip install -e .
python -c "import deep_ep"  # then relaunch

import importlib.util
if server_args.moe_a2a_backend == "deepep":
    assert importlib.util.find_spec("deep_ep") is not None, \
        "deepep backend selected but deep_ep not installed; install from deepseek-ai/deepep"

When it happens

Trigger: Launching with --moe-a2a-backend deepep (or EP MoE configs that default to deepep) on a node without the deepep Python package, or where deepep failed to import due to missing InfiniBand/NVSHMEM/compilation issues so the use_deepep flag ended up False.

Common situations: Running DeepSeek/V3-style EP inference on machines without the deepep wheel; deepep built for a different torch/CUDA/NVSHMEM version; docker images lacking deepep.

Related errors

• DeepEP returned FP8 input while Humming is configured for BF
• Humming expected DeepEP FP8 hidden states and group-128 scal
• Can't import trtllm_fp8_block_scale_moe from flashinfer. Ple
• Can't import trtllm_fp8_block_scale_routed_moe from flashinf
• aiter is required when SGLANG_USE_AITER is set to True

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6cb9fc02218c1bc9. Report an issue: GitHub.