{"record":{"id":"cc643c51448f6e7e","repo":"sgl-project/sglang","slug":"speculative-use-rejection-sampling-is-incompatib-cc643c","errorCode":null,"errorMessage":"--speculative-use-rejection-sampling is incompatible with --enable-deterministic-inference; the sampling kernel draws coins from the global RNG and is not batch-invariant.","messagePattern":"--speculative-use-rejection-sampling is incompatible with --enable-deterministic-inference; the sampling kernel draws coins from the global RNG and is not batch-invariant\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/speculative_hook.py","lineNumber":793,"sourceCode":"                \"EAGLE / EAGLE3 / NEXTN, not \"\n                f\"speculative_algorithm={cfg.speculative_algorithm}.\"\n            )\n        if cfg.speculative_eagle_topk != 1:\n            raise ValueError(\n                \"--speculative-use-rejection-sampling requires --speculative-eagle-topk=1.\"\n            )\n        if (\n            cfg.speculative_accept_threshold_single != 1.0\n            or cfg.speculative_accept_threshold_acc != 1.0\n        ):\n            raise ValueError(\n                \"--speculative-use-rejection-sampling is incompatible with \"\n                \"--speculative-accept-threshold-single / \"\n                \"--speculative-accept-threshold-acc; rejection sampling ignores \"\n                \"the accept thresholds.\"\n            )\n        if cfg.enable_deterministic_inference:\n            raise ValueError(\n                \"--speculative-use-rejection-sampling is incompatible with \"\n                \"--enable-deterministic-inference; the sampling kernel draws \"\n                \"coins from the global RNG and is not batch-invariant.\"\n            )\n\n        from sglang.srt.arg_groups.overrides import resolved_view\n\n        if (\n            resolved_view(server_args).enable_multi_layer_eagle\n            and cfg.speculative_eagle_topk != 1\n        ):\n            raise ValueError(\n                \"--speculative-use-rejection-sampling with multi-layer EAGLE \"\n                \"(--enable-multi-layer-eagle) requires --speculative-eagle-topk 1; \"\n                \"rejection sampling is only implemented for the linear (topk=1) chain.\"\n            )\n        logger.info(\n            \"Rejection sampling is enabled for speculative decoding \"","sourceCodeStart":775,"sourceCodeEnd":811,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/speculative_hook.py#L775-L811","documentation":"SGLang's --enable-deterministic-inference promises bit-identical output for identical batched requests. The rejection-sampling kernel consumes random coins drawn from the global RNG whose values depend on batching/timing, so its results are not batch-invariant and cannot honor determinism. The args hook rejects the combination with ValueError.","triggerScenarios":"Launching with both --speculative-use-rejection-sampling and --enable-deterministic-inference. The check runs after the topk/threshold validations in _handle_eagle_family.","commonSituations":"Enabling determinism for reproducible evals or regression testing on a server that also uses speculative decoding with rejection sampling; flipping a global determinism flag in a shared launch template without auditing speculative flags.","solutions":["Remove --enable-deterministic-inference when using rejection sampling","Or keep determinism and drop --speculative-use-rejection-sampling (standard speculative verify is still usable)","If approximate reproducibility suffices, fix seeds via sampling seeds instead of the determinism flag"],"exampleFix":"# before\n--speculative-use-rejection-sampling --enable-deterministic-inference\n# after\n--speculative-use-rejection-sampling","handlingStrategy":"validation","validationCode":"if server_args.speculative_use_rejection_sam_link := False: pass\nif server_args.speculative_use_rejection_sampling and server_args.enable_deterministic_inference:\n    server_args.enable_deterministic_inference = False  # determinism unsupported with RS","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep deterministic-eval and perf-tuned speculative configs in separate scripts","Remember RNG-based kernels (rejection sampling) break batch invariance"],"tags":["speculative-decoding","rejection-sampling","determinism","server-args"],"backgroundTag":"incompatible-launch-flags","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}