sgl-project/sglang · error · ValueError

DeepEP v2 MoE has not implemented the TBO/SBO overlap hooks

Error message

DeepEP v2 MoE has not implemented the TBO/SBO overlap hooks yet. Disable --enable-two-batch-overlap and --enable-single-batch-overlap when using --moe-a2a-backend deepep_v2.

What it means

Raised when deepep_v2 is enabled together with --enable-two-batch-overlap (TBO) or --enable-single-batch-overlap (SBO). DeepEP v2 has not implemented the overlap hooks that TBO/SBO require, so the combination cannot run.

Source

Thrown at python/sglang/srt/server_args.py:7724

                )
            # ElasticBuffer requires CUMEM, but not NVLS or its preallocation.
            os.environ.setdefault("NCCL_CUMEM_ENABLE", "1")
            # Respect model-level runner declarations before resolving auto.
            resolved_runner = resolved_view(self).moe_runner_backend
            if resolved_runner == "auto":
                self._declare("_handle_a2a_moe", moe_runner_backend="deep_gemm")
                logger.warning(
                    "DeepEP v2 MoE: resolved --moe-runner-backend auto -> deep_gemm."
                )
            elif resolved_runner != "deep_gemm":
                raise ValueError(
                    "DeepEP v2 MoE currently supports only "
                    f"--moe-runner-backend deep_gemm. Got {resolved_runner!r}. "
                    "Add a runner adapter before enabling DeepEP v2 with other "
                    "MoE runners."
                )
            if cfg.enable_two_batch_overlap or cfg.enable_single_batch_overlap:
                raise ValueError(
                    "DeepEP v2 MoE has not implemented the TBO/SBO overlap hooks yet. "
                    "Disable --enable-two-batch-overlap and "
                    "--enable-single-batch-overlap when using --moe-a2a-backend deepep_v2."
                )
            if cfg.enforce_shared_experts_fusion:
                raise ValueError(
                    "DeepEP v2 MoE has not validated fused shared experts yet. "
                    "Remove --enforce-shared-experts-fusion when using "
                    "--moe-a2a-backend deepep_v2."
                )
            # Prefill reads host counts and is not graph-capturable.
            self._declare(
                "_handle_a2a_moe",
                cuda_graph_config=with_phase(
                    cfg.cuda_graph_config, Phase.PREFILL, backend=Backend.DISABLED
                ),
            )
            logger.warning(

View on GitHub (pinned to 0132848349)

Solutions

1. Remove --enable-two-batch-overlap and --enable-single-batch-overlap from the launch flags
2. Keep TBO/SBO and stay on --moe-a2a-backend deepep until v2 overlap hooks land

Example fix

# before
--moe-a2a-backend deepep_v2 --enable-two-batch-overlap
# after
--moe-a2a-backend deepep_v2

if a2a_backend == "deepep_v2":
    flags.discard("--enable-two-batch-overlap")
    flags.discard("--enable-single-batch-overlap")

When it happens

Trigger: Launching with --moe-a2a-backend deepep_v2 and either --enable-two-batch-overlap or --enable-single-batch-overlap in the resolved config.

Common situations: Reusing a prefill-throughput-tuned config that enables TBO, then switching the a2a backend to deepep_v2; enabling SBO for latency while testing the v2 path.

Related errors

• DeepEP v2 MoE is not validated for {architecture!r}; support
• DeepEP v2 MoE is not validated as a speculative draft backen
• DeepEP v2 does not forward deterministic=True to ElasticBuff
• DeepEP v2 MoE currently supports only --moe-runner-backend d
• DeepEP v2 MoE has not validated fused shared experts yet. Re

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/69acd583d4797e1d. Report an issue: GitHub.