{"record":{"id":"95134b70e997e051","repo":"sgl-project/sglang","slug":"deepep-v2-does-not-forward-deterministic-true-to-e","errorCode":null,"errorMessage":"DeepEP v2 does not forward deterministic=True to ElasticBuffer, so deterministic sorting remains disabled. Disable --enable-deterministic-inference or use --moe-a2a-backend deepep.","messagePattern":"DeepEP v2 does not forward deterministic=True to ElasticBuffer, so deterministic sorting remains disabled\\. Disable --enable-deterministic-inference or use --moe-a2a-backend deepep\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":7701,"sourceCode":"            if cfg.deepep_mode == \"normal\":\n                logger.warning(\"Cuda graph is disabled because deepep_mode=`normal`\")\n                self._declare(\n                    \"_handle_a2a_moe\",\n                    cuda_graph_config=with_phase(\n                        cfg.cuda_graph_config, Phase.DECODE, backend=Backend.DISABLED\n                    ),\n                )\n                self._declare(\n                    \"_handle_a2a_moe\",\n                    cuda_graph_config=with_phase(\n                        cfg.cuda_graph_config, Phase.PREFILL, backend=Backend.DISABLED\n                    ),\n                )\n\n        if a2a_backend == \"deepep_v2\":\n            self._validate_deepep_v2_model_architecture()\n            if resolved_view(self).enable_deterministic_inference:\n                raise ValueError(\n                    \"DeepEP v2 does not forward deterministic=True to \"\n                    \"ElasticBuffer, so deterministic sorting remains disabled. \"\n                    \"Disable --enable-deterministic-inference or use \"\n                    \"--moe-a2a-backend deepep.\"\n                )\n            # ElasticBuffer requires CUMEM, but not NVLS or its preallocation.\n            os.environ.setdefault(\"NCCL_CUMEM_ENABLE\", \"1\")\n            # Respect model-level runner declarations before resolving auto.\n            resolved_runner = resolved_view(self).moe_runner_backend\n            if resolved_runner == \"auto\":\n                self._declare(\"_handle_a2a_moe\", moe_runner_backend=\"deep_gemm\")\n                logger.warning(\n                    \"DeepEP v2 MoE: resolved --moe-runner-backend auto -> deep_gemm.\"\n                )\n            elif resolved_runner != \"deep_gemm\":\n                raise ValueError(\n                    \"DeepEP v2 MoE currently supports only \"\n                    f\"--moe-runner-backend deep_gemm. Got {resolved_runner!r}. \"","sourceCodeStart":7683,"sourceCodeEnd":7719,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L7683-L7719","documentation":"Raised when --moe-a2a-backend deepep_v2 is combined with --enable-deterministic-inference. DeepEP v2's ElasticBuffer does not forward deterministic=True, so deterministic token sorting would silently stay disabled, breaking the determinism guarantee — so the resolver rejects the combo up front.","triggerScenarios":"Launching with --moe-a2a-backend deepep_v2 while --enable-deterministic-inference is set (enabled anywhere the resolved view can see it).","commonSituations":"Running reproducibility/regression test harnesses with deterministic inference enabled, then migrating the MoE path to deepep_v2 for bandwidth; CI configs that always set the determinism flag.","solutions":["Disable --enable-deterministic-inference when using deepep_v2","Fall back to --moe-a2a-backend deepep, which supports deterministic sorting"],"exampleFix":"# before\n--moe-a2a-backend deepep_v2 --enable-deterministic-inference\n# after\n--moe-a2a-backend deepep_v2\n# (or) --moe-a2a-backend deepep --enable-deterministic-inference","handlingStrategy":"validation","validationCode":"if a2a_backend == \"deepep_v2\" and enable_deterministic_inference:\n    a2a_backend = \"deepep\"  # determinism wins over v2 bandwidth","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat --enable-deterministic-inference as mutually exclusive with deepep_v2 in config templates","Gate determinism flags to test/CI profiles only"],"tags":["deepep","determinism","moe","server-args","elastic-buffer"],"backgroundTag":"unsupported-configuration-combo","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}