sgl-project/sglang · error · NotImplementedError

--speculative-use-rejection-sampling is only supported for E

Error message

--speculative-use-rejection-sampling is only supported for EAGLE / EAGLE3 / NEXTN, not speculative_algorithm={}.

What it means

SGLang's --speculative-use-rejection-sampling flag routes speculative verification through a rejection-sampling kernel that only exists for the EAGLE-family draft workers (EAGLE, EAGLE3, and NEXTN which resolves to EAGLE). Only those workers emit a target-vocab proposal distribution the kernel can compare against. The server-args hook raises NotImplementedError when any other speculative algorithm (STANDALONE, FROZEN_KV_MTP, NGRAM, DFLASH) is combined with this flag.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:773

            "_handle_eagle_family.auto_params",
            speculative_num_steps=steps,
            speculative_eagle_topk=topk,
            speculative_num_draft_tokens=draft_tokens,
        )

    if "trtllm_mha" in attention_backends_of(resolved_view(server_args)):
        if cfg.speculative_eagle_topk > 1:
            raise ValueError(
                "trtllm_mha backend only supports topk = 1 for speculative decoding."
            )

    if cfg.speculative_use_rejection_sampling:
        # Resolved alias by now: NEXTN -> EAGLE, Gemma4 draft -> FROZEN_KV_MTP.
        # Only the EAGLE/EAGLE3 draft workers emit a target-vocab proposal that
        # the rejection-sampling kernel consumes; everything else (STANDALONE,
        # FROZEN_KV_MTP, NGRAM, DFLASH) is unsupported.
        if cfg.speculative_algorithm not in ("EAGLE", "EAGLE3"):
            raise NotImplementedError(
                "--speculative-use-rejection-sampling is only supported for "
                "EAGLE / EAGLE3 / NEXTN, not "
                f"speculative_algorithm={cfg.speculative_algorithm}."
            )
        if cfg.speculative_eagle_topk != 1:
            raise ValueError(
                "--speculative-use-rejection-sampling requires --speculative-eagle-topk=1."
            )
        if (
            cfg.speculative_accept_threshold_single != 1.0
            or cfg.speculative_accept_threshold_acc != 1.0
        ):
            raise ValueError(
                "--speculative-use-rejection-sampling is incompatible with "
                "--speculative-accept-threshold-single / "
                "--speculative-accept-threshold-acc; rejection sampling ignores "
                "the accept thresholds."
            )

View on GitHub (pinned to 0132848349)

Solutions

Set --speculative-algorithm EAGLE or EAGLE3 (NEXTN also resolves to EAGLE) if you want rejection sampling
Remove --speculative-use-rejection-sampling if you must use NGRAM/STANDALONE/FROZEN_KV_MTP/DFLASH — they fall back to the standard tree-based verify path
If you wanted NEXTN behavior, keep --speculative-algorithm NEXTN (alias resolves to EAGLE) rather than a FROZEN_KV_MTP/Gemma4 draft path

Example fix

# before
--speculative-algorithm NGRAM --speculative-use-rejection-sampling
# after
--speculative-algorithm NGRAM   # rejection sampling dropped

Defensive patterns

Strategy: validation

Validate before calling

from sglang.srt.arg_groups.speculative_hook import resolving_view
algo = resolving_view(server_args).speculative_algorithm
if server_args.speculative_use_rejection_sampling and algo not in ("EAGLE", "EAGLE3", "NEXTN"):
    # drop the flag or switch algorithm before launch
    server_args.speculative_use_rejection_sampling = False

Prevention

Keep one launch script per speculative algorithm instead of mixing flags
Assert the algorithm is EAGLE/EAGLE3/NEXTN before setting --speculative-use-rejection-sampling

When it happens

Trigger: Launching the server with --speculative-use-rejection-sampling together with --speculative-algorithm set to NGRAM, STANDALONE, FROZEN_KV_MTP, or DFLASH. Note NEXTN and Gemma4-draft aliases are resolved before this check, so NEXTN is accepted while its resolved forms like FROZEN_KV_MTP are not.

Common situations: Copying an EAGLE launch command but swapping in --speculative-algorithm NGRAM or a MTP variant; enabling a new MTP-style algorithm (e.g. a frozen-KV draft model) and assuming rejection sampling carries over; upgrading SGLang where a previously accepted draft backend was renamed/reclassified.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e6968f0acfa994de. Report an issue: GitHub.