sgl-project/sglang · error · ValueError

DeepEP v2 does not forward deterministic=True to ElasticBuff

Error message

DeepEP v2 does not forward deterministic=True to ElasticBuffer, so deterministic sorting remains disabled. Disable --enable-deterministic-inference or use --moe-a2a-backend deepep.

What it means

Raised when --moe-a2a-backend deepep_v2 is combined with --enable-deterministic-inference. DeepEP v2's ElasticBuffer does not forward deterministic=True, so deterministic token sorting would silently stay disabled, breaking the determinism guarantee — so the resolver rejects the combo up front.

Source

Thrown at python/sglang/srt/server_args.py:7701

            if cfg.deepep_mode == "normal":
                logger.warning("Cuda graph is disabled because deepep_mode=`normal`")
                self._declare(
                    "_handle_a2a_moe",
                    cuda_graph_config=with_phase(
                        cfg.cuda_graph_config, Phase.DECODE, backend=Backend.DISABLED
                    ),
                )
                self._declare(
                    "_handle_a2a_moe",
                    cuda_graph_config=with_phase(
                        cfg.cuda_graph_config, Phase.PREFILL, backend=Backend.DISABLED
                    ),
                )

        if a2a_backend == "deepep_v2":
            self._validate_deepep_v2_model_architecture()
            if resolved_view(self).enable_deterministic_inference:
                raise ValueError(
                    "DeepEP v2 does not forward deterministic=True to "
                    "ElasticBuffer, so deterministic sorting remains disabled. "
                    "Disable --enable-deterministic-inference or use "
                    "--moe-a2a-backend deepep."
                )
            # ElasticBuffer requires CUMEM, but not NVLS or its preallocation.
            os.environ.setdefault("NCCL_CUMEM_ENABLE", "1")
            # Respect model-level runner declarations before resolving auto.
            resolved_runner = resolved_view(self).moe_runner_backend
            if resolved_runner == "auto":
                self._declare("_handle_a2a_moe", moe_runner_backend="deep_gemm")
                logger.warning(
                    "DeepEP v2 MoE: resolved --moe-runner-backend auto -> deep_gemm."
                )
            elif resolved_runner != "deep_gemm":
                raise ValueError(
                    "DeepEP v2 MoE currently supports only "
                    f"--moe-runner-backend deep_gemm. Got {resolved_runner!r}. "

View on GitHub (pinned to 0132848349)

Solutions

1. Disable --enable-deterministic-inference when using deepep_v2
2. Fall back to --moe-a2a-backend deepep, which supports deterministic sorting

Example fix

# before
--moe-a2a-backend deepep_v2 --enable-deterministic-inference
# after
--moe-a2a-backend deepep_v2
# (or) --moe-a2a-backend deepep --enable-deterministic-inference

if a2a_backend == "deepep_v2" and enable_deterministic_inference:
    a2a_backend = "deepep"  # determinism wins over v2 bandwidth

When it happens

Trigger: Launching with --moe-a2a-backend deepep_v2 while --enable-deterministic-inference is set (enabled anywhere the resolved view can see it).

Common situations: Running reproducibility/regression test harnesses with deterministic inference enabled, then migrating the MoE path to deepep_v2 for bandwidth; CI configs that always set the determinism flag.

Related errors

• DeepEP v2 MoE is not validated for {architecture!r}; support
• DeepEP v2 MoE is not validated as a speculative draft backen
• DeepEP v2 MoE currently supports only --moe-runner-backend d
• DeepEP v2 MoE has not implemented the TBO/SBO overlap hooks
• DeepEP v2 MoE has not validated fused shared experts yet. Re

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/95134b70e997e051. Report an issue: GitHub.