sgl-project/sglang · error · ValueError

--speculative-use-rejection-sampling is incompatible with --

Error message

--speculative-use-rejection-sampling is incompatible with --speculative-accept-threshold-single / --speculative-accept-threshold-acc; rejection sampling ignores the accept thresholds.

What it means

Rejection sampling already guarantees exact sampling from the target distribution — every proposal is accepted/rejected with the exact target probability, so accept-threshold heuristics are meaningless. SGLang therefore refuses the combination of --speculative-use-rejection-sampling with non-default --speculative-accept-threshold-single or --speculative-accept-threshold-acc (both default 1.0) rather than silently ignoring them.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:786

        # Resolved alias by now: NEXTN -> EAGLE, Gemma4 draft -> FROZEN_KV_MTP.
        # Only the EAGLE/EAGLE3 draft workers emit a target-vocab proposal that
        # the rejection-sampling kernel consumes; everything else (STANDALONE,
        # FROZEN_KV_MTP, NGRAM, DFLASH) is unsupported.
        if cfg.speculative_algorithm not in ("EAGLE", "EAGLE3"):
            raise NotImplementedError(
                "--speculative-use-rejection-sampling is only supported for "
                "EAGLE / EAGLE3 / NEXTN, not "
                f"speculative_algorithm={cfg.speculative_algorithm}."
            )
        if cfg.speculative_eagle_topk != 1:
            raise ValueError(
                "--speculative-use-rejection-sampling requires --speculative-eagle-topk=1."
            )
        if (
            cfg.speculative_accept_threshold_single != 1.0
            or cfg.speculative_accept_threshold_acc != 1.0
        ):
            raise ValueError(
                "--speculative-use-rejection-sampling is incompatible with "
                "--speculative-accept-threshold-single / "
                "--speculative-accept-threshold-acc; rejection sampling ignores "
                "the accept thresholds."
            )
        if cfg.enable_deterministic_inference:
            raise ValueError(
                "--speculative-use-rejection-sampling is incompatible with "
                "--enable-deterministic-inference; the sampling kernel draws "
                "coins from the global RNG and is not batch-invariant."
            )

        from sglang.srt.arg_groups.overrides import resolved_view

        if (
            resolved_view(server_args).enable_multi_layer_eagle
            and cfg.speculative_eagle_topk != 1
        ):

View on GitHub (pinned to 0132848349)

Solutions

  1. Remove the --speculative-accept-threshold-single / --speculative-accept-threshold-acc overrides (let them default to 1.0)
  2. Or explicitly set both to 1.0 alongside rejection sampling
  3. If you want threshold-based relaxed acceptance, drop --speculative-use-rejection-sampling instead

Example fix

# before
--speculative-use-rejection-sampling \
  --speculative-accept-threshold-single 0.9 \
  --speculative-accept-threshold-acc 0.8
# after
--speculative-use-rejection-sampling  # thresholds left at default 1.0
Defensive patterns

Strategy: validation

Validate before calling

if server_args.speculative_use_rejection_sampling:
    server_args.speculative_accept_threshold_single = 1.0
    server_args.speculative_accept_threshold_acc = 1.0

Prevention

When it happens

Trigger: Launching with --speculative-use-rejection-sampling plus --speculative-accept-threshold-single != 1.0 or --speculative-accept-threshold-acc != 1.0. Checked in _handle_eagle_family during handle_server_args.

Common situations: Tuning relaxed-acceptance thresholds (e.g. 0.95 / 0.85) for higher acceptance rate on a normal EAGLE setup and then flipping on rejection sampling in the same script; carrying over tuned threshold values in a shared config file.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e0010de948aa50ef. Report an issue: GitHub.