sgl-project/sglang · error · ValueError

--speculative-use-rejection-sampling requires --speculative-

Error message

--speculative-use-rejection-sampling requires --speculative-eagle-topk=1.

What it means

Rejection sampling in SGLang is only implemented for the linear (topk=1) draft chain: with a single draft hypothesis the verifier can do an exact accept/reject coin flip against the target distribution. With --speculative-eagle-topk > 1 there are multiple draft branches and the rejection-sampling kernel does not support that shape, so the args hook rejects the combination with a ValueError.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:779

    if "trtllm_mha" in attention_backends_of(resolved_view(server_args)):
        if cfg.speculative_eagle_topk > 1:
            raise ValueError(
                "trtllm_mha backend only supports topk = 1 for speculative decoding."
            )

    if cfg.speculative_use_rejection_sampling:
        # Resolved alias by now: NEXTN -> EAGLE, Gemma4 draft -> FROZEN_KV_MTP.
        # Only the EAGLE/EAGLE3 draft workers emit a target-vocab proposal that
        # the rejection-sampling kernel consumes; everything else (STANDALONE,
        # FROZEN_KV_MTP, NGRAM, DFLASH) is unsupported.
        if cfg.speculative_algorithm not in ("EAGLE", "EAGLE3"):
            raise NotImplementedError(
                "--speculative-use-rejection-sampling is only supported for "
                "EAGLE / EAGLE3 / NEXTN, not "
                f"speculative_algorithm={cfg.speculative_algorithm}."
            )
        if cfg.speculative_eagle_topk != 1:
            raise ValueError(
                "--speculative-use-rejection-sampling requires --speculative-eagle-topk=1."
            )
        if (
            cfg.speculative_accept_threshold_single != 1.0
            or cfg.speculative_accept_threshold_acc != 1.0
        ):
            raise ValueError(
                "--speculative-use-rejection-sampling is incompatible with "
                "--speculative-accept-threshold-single / "
                "--speculative-accept-threshold-acc; rejection sampling ignores "
                "the accept thresholds."
            )
        if cfg.enable_deterministic_inference:
            raise ValueError(
                "--speculative-use-rejection-sampling is incompatible with "
                "--enable-deterministic-inference; the sampling kernel draws "
                "coins from the global RNG and is not batch-invariant."
            )

View on GitHub (pinned to 0132848349)

Solutions

Set --speculative-eagle-topk 1 and keep --speculative-use-rejection-sampling
Or drop --speculative-use-rejection-sampling and keep tree-based topk > 1 (uses the standard verify path)
If you need both tree drafting and exact sampling, wait for/wrap a kernel that supports topk>1; not currently available

Example fix

# before
--speculative-use-rejection-sampling --speculative-eagle-topk 4
# after
--speculative-use-rejection-sampling --speculative-eagle-topk 1

Defensive patterns

Strategy: validation

Validate before calling

if server_args.speculative_use_rejection_sampling:
    assert server_args.speculative_eagle_topk == 1, "rejection sampling requires topk=1"

Prevention

Treat topk=1 as part of the rejection-sampling preset, not an independent knob
Document tuned EAGLE configs with which verify path they target

When it happens

Trigger: Launching with --speculative-use-rejection-sampling and --speculative-eagle-topk set to a value other than 1 (e.g. 4 or 8 for tree-based drafting). The check fires during handle_server_args validation in _handle_eagle_family.

Common situations: Copying a tree-based EAGLE config (topk=4/8) from a benchmark and adding the rejection-sampling flag; tuning topk for throughput and forgetting the flag constrains it to 1.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/83dda4add75b0c7b. Report an issue: GitHub.