sgl-project/sglang · error · ValueError

--speculative-use-rejection-sampling with multi-layer EAGLE

Error message

--speculative-use-rejection-sampling with multi-layer EAGLE (--enable-multi-layer-eagle) requires --speculative-eagle-topk 1; rejection sampling is only implemented for the linear (topk=1) chain.

What it means

Multi-layer EAGLE (draft chains with several verification layers) is only compatible with the rejection-sampling kernel when the draft tree is a linear chain, i.e. --speculative-eagle-topk=1. With topk>1 each layer branches into a tree and the rejection-sampling implementation cannot handle it. The check uses the resolved-view value of enable_multi_layer_eagle (set via model overrides), so even indirect enablement is caught.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:805

                "--speculative-use-rejection-sampling is incompatible with "
                "--speculative-accept-threshold-single / "
                "--speculative-accept-threshold-acc; rejection sampling ignores "
                "the accept thresholds."
            )
        if cfg.enable_deterministic_inference:
            raise ValueError(
                "--speculative-use-rejection-sampling is incompatible with "
                "--enable-deterministic-inference; the sampling kernel draws "
                "coins from the global RNG and is not batch-invariant."
            )

        from sglang.srt.arg_groups.overrides import resolved_view

        if (
            resolved_view(server_args).enable_multi_layer_eagle
            and cfg.speculative_eagle_topk != 1
        ):
            raise ValueError(
                "--speculative-use-rejection-sampling with multi-layer EAGLE "
                "(--enable-multi-layer-eagle) requires --speculative-eagle-topk 1; "
                "rejection sampling is only implemented for the linear (topk=1) chain."
            )
        logger.info(
            "Rejection sampling is enabled for speculative decoding "
            "(speculative_use_rejection_sampling=True)."
        )

    if (
        cfg.speculative_eagle_topk == 1
        and cfg.speculative_num_draft_tokens != cfg.speculative_num_steps + 1
    ):
        logger.warning(
            "speculative_num_draft_tokens is adjusted to speculative_num_steps + 1 when speculative_eagle_topk == 1"
        )
        declare_resolution(
            server_args,

View on GitHub (pinned to 0132848349)

Solutions

1. Set --speculative-eagle-topk 1 when using multi-layer EAGLE with rejection sampling
2. Or disable multi-layer EAGLE (remove --enable-multi-layer-eagle / the model override) and keep topk as-is
3. Or drop --speculative-use-rejection-sampling to use the tree verify path

Example fix

# before
--speculative-use-rejection-sampling --enable-multi-layer-eagle --speculative-eagle-topk 4
# after
--speculative-use-rejection-sampling --enable-multi-layer-eagle --speculative-eagle-topk 1

from sglang.srt.arg_groups.overrides import resolved_view
if server_args.speculative_use_rejection_sampling:
    v = resolved_view(server_args)
    assert not v.enable_multi_layer_eagle or server_args.speculative_eagle_topk == 1

When it happens

Trigger: Launching with --speculative-use-rejection-sampling while enable_multi_layer_eagle resolves true (CLI flag or model-override) and --speculative-eagle-topk != 1. Fires after the earlier topk==1 check is bypassed via override layers.

Common situations: Using a model whose server-args overrides enable multi-layer EAGLE by default (e.g. via resolved_view overrides) combined with a copied EAGLE topk>1 config and rejection sampling; changing topk after a multi-layer setup was working.

Related errors

• --speculative-use-rejection-sampling is only supported for E
• --speculative-use-rejection-sampling requires --speculative-
• --speculative-use-rejection-sampling is incompatible with --
• --speculative-use-rejection-sampling is incompatible with --
• Currently DFLASH speculative decoding does not support dp at

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5d6f7698daddd1ec. Report an issue: GitHub.