sgl-project/sglang · error · ValueError

--speculative-use-rejection-sampling is incompatible with --

Error message

--speculative-use-rejection-sampling is incompatible with --enable-deterministic-inference; the sampling kernel draws coins from the global RNG and is not batch-invariant.

What it means

SGLang's --enable-deterministic-inference promises bit-identical output for identical batched requests. The rejection-sampling kernel consumes random coins drawn from the global RNG whose values depend on batching/timing, so its results are not batch-invariant and cannot honor determinism. The args hook rejects the combination with ValueError.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:793

                "EAGLE / EAGLE3 / NEXTN, not "
                f"speculative_algorithm={cfg.speculative_algorithm}."
            )
        if cfg.speculative_eagle_topk != 1:
            raise ValueError(
                "--speculative-use-rejection-sampling requires --speculative-eagle-topk=1."
            )
        if (
            cfg.speculative_accept_threshold_single != 1.0
            or cfg.speculative_accept_threshold_acc != 1.0
        ):
            raise ValueError(
                "--speculative-use-rejection-sampling is incompatible with "
                "--speculative-accept-threshold-single / "
                "--speculative-accept-threshold-acc; rejection sampling ignores "
                "the accept thresholds."
            )
        if cfg.enable_deterministic_inference:
            raise ValueError(
                "--speculative-use-rejection-sampling is incompatible with "
                "--enable-deterministic-inference; the sampling kernel draws "
                "coins from the global RNG and is not batch-invariant."
            )

        from sglang.srt.arg_groups.overrides import resolved_view

        if (
            resolved_view(server_args).enable_multi_layer_eagle
            and cfg.speculative_eagle_topk != 1
        ):
            raise ValueError(
                "--speculative-use-rejection-sampling with multi-layer EAGLE "
                "(--enable-multi-layer-eagle) requires --speculative-eagle-topk 1; "
                "rejection sampling is only implemented for the linear (topk=1) chain."
            )
        logger.info(
            "Rejection sampling is enabled for speculative decoding "

View on GitHub (pinned to 0132848349)

Solutions

1. Remove --enable-deterministic-inference when using rejection sampling
2. Or keep determinism and drop --speculative-use-rejection-sampling (standard speculative verify is still usable)
3. If approximate reproducibility suffices, fix seeds via sampling seeds instead of the determinism flag

Example fix

# before
--speculative-use-rejection-sampling --enable-deterministic-inference
# after
--speculative-use-rejection-sampling

if server_args.speculative_use_rejection_sam_link := False: pass
if server_args.speculative_use_rejection_sampling and server_args.enable_deterministic_inference:
    server_args.enable_deterministic_inference = False  # determinism unsupported with RS

When it happens

Trigger: Launching with both --speculative-use-rejection-sampling and --enable-deterministic-inference. The check runs after the topk/threshold validations in _handle_eagle_family.

Common situations: Enabling determinism for reproducible evals or regression testing on a server that also uses speculative decoding with rejection sampling; flipping a global determinism flag in a shared launch template without auditing speculative flags.

Related errors

• --speculative-use-rejection-sampling is only supported for E
• --speculative-use-rejection-sampling requires --speculative-
• --speculative-use-rejection-sampling is incompatible with --
• --speculative-use-rejection-sampling with multi-layer EAGLE
• Currently DFLASH speculative decoding does not support dp at

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/cc643c51448f6e7e. Report an issue: GitHub.