sgl-project/sglang · error · ValueError

speculative_eagle_topk > 1 with page_size > 1 is only suppor

Error message

speculative_eagle_topk > 1 with page_size > 1 is only supported on {_PAGE_TREE_SPEC_BACKENDS}; got attention_backend={view.attention_backend!r}. Use page_size == 1 or one of those backends.

What it means

Tree-based EAGLE drafting (speculative_eagle_topk > 1) requires the attention backend to support paged KV for a draft tree when page_size > 1. Only backends in _PAGE_TREE_SPEC_BACKENDS implement tree-shaped page tables; others (e.g. FlashInfer variants without tree paging) cannot store the branching draft tokens under multi-token pages, so the combination is rejected.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:838

            "speculative_num_draft_tokens is adjusted to speculative_num_steps + 1 when speculative_eagle_topk == 1"
        )
        declare_resolution(
            server_args,
            "_handle_eagle_family",
            speculative_num_draft_tokens=cfg.speculative_num_steps + 1,
        )

    # topk > 1 + page_size > 1 needs the two-pass cascade draft-decode (shared prefix
    # pass + per-branch expand pass with prefix-tail dup). Only these backends implement
    # it; flashmla / trtllm_mla / cutlass_mla can't express the per-branch tree, so reject.
    _PAGE_TREE_SPEC_BACKENDS = ("flashinfer", "fa3", "triton")
    view = resolved_view(server_args)
    if (
        cfg.speculative_eagle_topk > 1
        and view.page_size > 1
        and view.attention_backend not in _PAGE_TREE_SPEC_BACKENDS
    ):
        raise ValueError(
            f"speculative_eagle_topk > 1 with page_size > 1 is only supported on "
            f"{_PAGE_TREE_SPEC_BACKENDS}; got attention_backend="
            f"{view.attention_backend!r}. Use page_size == 1 or one of those backends."
        )


def _handle_ngram(server_args: ServerArgs) -> None:
    cfg = resolving_view(server_args)
    if cfg.device not in ("cuda", "cpu"):
        raise ValueError(
            "Ngram speculative decoding only supports CUDA or CPU devices."
        )

    _disable_overlap_schedule_for_cpu(server_args)

    if cfg.max_running_requests is None:
        declare_resolution(
            server_args,

View on GitHub (pinned to 0132848349)

Solutions

1. Set --page-size 1
2. Or switch to one of the tree-capable backends listed in _PAGE_TREE_SPEC_BACKENDS (check speculative_hook.py for the current list)
3. Or reduce --speculative-eagle-topk to 1 (linear chain works on any backend)

Example fix

# before
--speculative-eagle-topk 8 --page-size 64 --attention-backend triton
# after
--speculative-eagle-topk 8 --page-size 1

from sglang.srt.arg_groups.speculative_hook import _PAGE_TREE_SPEC_BACKENDS
from sglang.srt.arg_groups.overrides import resolved_view
v = resolved_view(server_args)
if server_args.speculative_eagle_topk > 1 and v.page_size > 1:
    assert v.attention_backend in _PAGE_TREE_SPEC_BACKENDS, "tree spec + page>1 needs tree-capable backend"

When it happens

Trigger: Launching with speculative EAGLE where speculative_eagle_topk > 1 and the resolved page_size > 1 while attention_backend resolves to a backend not in _PAGE_TREE_SPEC_BACKENDS (e.g. some FlashMLA/FlashInfer/Triton configs).

Common situations: Forcing an attention backend for perf tuning (e.g. --attention-backend triton) on a page-size>1 setup while keeping tree drafting; using DP/TP hardware presets that default to a non-tree backend with page_size>1; recent SGLang versions that added page_size>1 defaults.

Related errors

• --speculative-use-rejection-sampling requires --speculative-
• MiniMax-H3 ring parallelism requires the FlashAttention back
• Currently DFLASH speculative decoding does not support dp at
• Currently DFLASH speculative decoding only supports pp_size
• trtllm_mha backend only supports topk = 1 for speculative de

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8840061d8cca8e01. Report an issue: GitHub.