sgl-project/sglang · error · ValueError

DeepEP v2 MoE currently supports only --moe-runner-backend d

Error message

DeepEP v2 MoE currently supports only --moe-runner-backend deep_gemm. Got {resolved_runner!r}. Add a runner adapter before enabling DeepEP v2 with other MoE runners.

What it means

Raised when deepep_v2 is used with a --moe-runner-backend other than deep_gemm. Only the deep_gemm runner has been adapted to DeepEP v2's dispatch/combine contract; 'auto' is silently coerced to deep_gemm, but any other explicit runner (triton, flashinfer_trtllm, etc.) is rejected.

Source

Thrown at python/sglang/srt/server_args.py:7717

            self._validate_deepep_v2_model_architecture()
            if resolved_view(self).enable_deterministic_inference:
                raise ValueError(
                    "DeepEP v2 does not forward deterministic=True to "
                    "ElasticBuffer, so deterministic sorting remains disabled. "
                    "Disable --enable-deterministic-inference or use "
                    "--moe-a2a-backend deepep."
                )
            # ElasticBuffer requires CUMEM, but not NVLS or its preallocation.
            os.environ.setdefault("NCCL_CUMEM_ENABLE", "1")
            # Respect model-level runner declarations before resolving auto.
            resolved_runner = resolved_view(self).moe_runner_backend
            if resolved_runner == "auto":
                self._declare("_handle_a2a_moe", moe_runner_backend="deep_gemm")
                logger.warning(
                    "DeepEP v2 MoE: resolved --moe-runner-backend auto -> deep_gemm."
                )
            elif resolved_runner != "deep_gemm":
                raise ValueError(
                    "DeepEP v2 MoE currently supports only "
                    f"--moe-runner-backend deep_gemm. Got {resolved_runner!r}. "
                    "Add a runner adapter before enabling DeepEP v2 with other "
                    "MoE runners."
                )
            if cfg.enable_two_batch_overlap or cfg.enable_single_batch_overlap:
                raise ValueError(
                    "DeepEP v2 MoE has not implemented the TBO/SBO overlap hooks yet. "
                    "Disable --enable-two-batch-overlap and "
                    "--enable-single-batch-overlap when using --moe-a2a-backend deepep_v2."
                )
            if cfg.enforce_shared_experts_fusion:
                raise ValueError(
                    "DeepEP v2 MoE has not validated fused shared experts yet. "
                    "Remove --enforce-shared-experts-fusion when using "
                    "--moe-a2a-backend deepep_v2."
                )
            # Prefill reads host counts and is not graph-capturable.

View on GitHub (pinned to 0132848349)

Solutions

1. Remove the --moe-runner-backend override so 'auto' resolves to deep_gemm
2. Set --moe-runner-backend deep_gemm explicitly
3. If you need a different runner, switch to --moe-a2a-backend deepep until an adapter is added

Example fix

# before
--moe-a2a-backend deepep_v2 --moe-runner-backend flashinfer_trtllm
# after
--moe-a2a-backend deepep_v2 --moe-runner-backend deep_gemm

if a2a_backend == "deepep_v2" and moe_runner_backend not in (None, "auto", "deep_gemm"):
    raise SystemExit(f"deepep_v2 supports only deep_gemm runner, got {moe_runner_backend}")

When it happens

Trigger: Launching with --moe-a2a-backend deepep_v2 and an explicit --moe-runner-backend like triton or flashinfer_trtllm.

Common situations: Porting a tuned FP8/FP4 serving config that pinned a specific runner (e.g. flashinfer for FP4) onto a deepep_v2 deployment; overriding the runner per-environment without knowing the v2 restriction.

Related errors

• DeepEP v2 MoE is not validated for {architecture!r}; support
• DeepEP v2 MoE is not validated as a speculative draft backen
• DeepEP v2 does not forward deterministic=True to ElasticBuff
• DeepEP v2 MoE has not implemented the TBO/SBO overlap hooks
• DeepEP v2 MoE has not validated fused shared experts yet. Re

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1a09d4d99c8f2dd6. Report an issue: GitHub.