sgl-project/sglang · error · ValueError

--enable-linear-replayssm-spec requires a linear draft chain

Error message

--enable-linear-replayssm-spec requires a linear draft chain (--speculative-eagle-topk in {None, 1}); the chunked verify kernel uses a strictly-lower causal mask and is invalid for EAGLE tree verify. Got --speculative-eagle-topk={cfg.speculative_eagle_topk!r}.

What it means

ReplaySSM spec-verify (--enable-linear-replayssm-spec) only supports a linear draft chain, i.e. --speculative-eagle-topk in {None, 1}. The chunked verify kernel uses a strictly-lower causal mask, which is invalid for EAGLE tree verification with topk > 1; tree topologies must fall back to the recurrent verify.

Source

Thrown at python/sglang/srt/server_args.py:6979

            if cfg.linear_replayssm_cache_len < 1:
                raise ValueError(
                    "--linear-replayssm-cache-len must be >= 1, got "
                    f"{cfg.linear_replayssm_cache_len}."
                )

        # ReplaySSM spec-verify (Part B of #28511): linear-chain target verify via
        # fold-every-commit -- the verify stores each draft step's raw inputs into
        # the per-slot (rawv, rawk, g, beta) window and the commit replays the
        # accepted prefix into the fp32 checkpoint. The intra-window interaction
        # uses a strictly-lower causal mask, so it is valid ONLY for a linear
        # draft chain (speculative_eagle_topk in {None, 1}, i.e. NEXTN / MTP);
        # EAGLE tree verify (topk > 1) must fall back to the recurrent verify.
        # GDN sizes the window to the draft maximum; KDA (kda_backend) keeps a
        # --linear-replayssm-cache-len window and folds via its own fused
        # verify ring-write + commit_kda_replayssm_after_verify.
        if cfg.enable_linear_replayssm_spec:
            if cfg.speculative_eagle_topk not in (None, 1):
                raise ValueError(
                    "--enable-linear-replayssm-spec requires a linear draft chain "
                    "(--speculative-eagle-topk in {None, 1}); the chunked verify "
                    "kernel uses a strictly-lower causal mask and is invalid for "
                    "EAGLE tree verify. Got "
                    f"--speculative-eagle-topk={cfg.speculative_eagle_topk!r}."
                )
            if decode not in ("triton", "flashinfer"):
                raise ValueError(
                    "--enable-linear-replayssm-spec requires the triton or "
                    "flashinfer linear-attn decode backend, got "
                    f"--linear-attn-decode-backend={decode!r}."
                )
            from sglang.srt.speculative.ragged_verify import (
                RaggedVerifyMode,
                read_ragged_verify_mode,
            )

            ragged_mode = read_ragged_verify_mode()

View on GitHub (pinned to 0132848349)

Solutions

  1. Set --speculative-eagle-topk 1 (linear chain) or remove the flag
  2. Or disable --enable-linear-replayssm-spec and use the recurrent verify path for tree drafting

Example fix

# before
--enable-linear-replayssm-spec --speculative-eagle-topk 4
# after
--enable-linear-replayssm-spec --speculative-eagle-topk 1
Defensive patterns

Strategy: validation

Validate before calling

def validate(topk, enable_spec):
    return not enable_spec or topk in (None, 1)

Type guard

def is_linear_chain(topk) -> bool: return topk in (None, 1)

Prevention

When it happens

Trigger: --enable-linear-replayssm-spec with --speculative-eagle-topk 2+ (EAGLE tree drafting) on a hybrid linear-attention model.

Common situations: Reusing an EAGLE topk>1 spec-decode config from a full-attention model when enabling ReplaySSM spec on GDN/KDA hybrids; not realizing the spec path is chain-only.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/12f769e13fc97f9a. Report an issue: GitHub.