sgl-project/sglang · error · ValueError

--enable-linear-replayssm-spec requires the triton or flashi

Error message

--enable-linear-replayssm-spec requires the triton or flashinfer linear-attn decode backend, got --linear-attn-decode-backend={decode!r}.

What it means

ReplaySSM spec-verify requires the triton or flashinfer linear-attention decode backend. Note this differs from base ReplaySSM (errors 6042), which allows only triton/helion — the spec-verify chunked kernel is implemented for triton and flashinfer decode paths.

Source

Thrown at python/sglang/srt/server_args.py:6987

        # the per-slot (rawv, rawk, g, beta) window and the commit replays the
        # accepted prefix into the fp32 checkpoint. The intra-window interaction
        # uses a strictly-lower causal mask, so it is valid ONLY for a linear
        # draft chain (speculative_eagle_topk in {None, 1}, i.e. NEXTN / MTP);
        # EAGLE tree verify (topk > 1) must fall back to the recurrent verify.
        # GDN sizes the window to the draft maximum; KDA (kda_backend) keeps a
        # --linear-replayssm-cache-len window and folds via its own fused
        # verify ring-write + commit_kda_replayssm_after_verify.
        if cfg.enable_linear_replayssm_spec:
            if cfg.speculative_eagle_topk not in (None, 1):
                raise ValueError(
                    "--enable-linear-replayssm-spec requires a linear draft chain "
                    "(--speculative-eagle-topk in {None, 1}); the chunked verify "
                    "kernel uses a strictly-lower causal mask and is invalid for "
                    "EAGLE tree verify. Got "
                    f"--speculative-eagle-topk={cfg.speculative_eagle_topk!r}."
                )
            if decode not in ("triton", "flashinfer"):
                raise ValueError(
                    "--enable-linear-replayssm-spec requires the triton or "
                    "flashinfer linear-attn decode backend, got "
                    f"--linear-attn-decode-backend={decode!r}."
                )
            from sglang.srt.speculative.ragged_verify import (
                RaggedVerifyMode,
                read_ragged_verify_mode,
            )

            ragged_mode = read_ragged_verify_mode()
            if ragged_mode is not RaggedVerifyMode.STATIC:
                # Ragged ring-writes need the KDA fold-every-commit family
                # (DSPARK/DFLASH) + the triton verify kernel (nv_cutedsl falls
                # back to it for ragged layouts). The GDN ring-write kernels do
                # not take the ragged layout and the flashinfer verify kernel
                # never writes the ring -> a stale ring would be folded; keep
                # refusing those combinations.
                _algo = (cfg.speculative_algorithm or "").upper()

View on GitHub (pinned to 0132848349)

Solutions

  1. Set --linear-attn-decode-backend triton (or flashinfer where supported)
  2. Or drop --enable-linear-replayssm-spec while keeping helion decode for base ReplaySSM

Example fix

# before
--enable-linear-replayssm-spec --linear-attn-decode-backend helion
# after
--enable-linear-replayssm-spec --linear-attn-decode-backend triton
Defensive patterns

Strategy: validation

Validate before calling

SPEC_DECODE_OK = {"triton", "flashinfer"}
def validate(decode_backend, enable_spec):
    return not enable_spec or decode_backend in SPEC_DECODE_OK

Type guard

def spec_compatible_decode(b: str) -> bool: return b in {"triton", "flashinfer"}

Prevention

When it happens

Trigger: --enable-linear-replayssm-spec with --linear-attn-decode-backend helion (or any value outside {triton, flashinfer}).

Common situations: Running KDA with helion decode (valid for base ReplaySSM) and additionally enabling the spec-verify flag; mixing the two flag sets' backend requirements.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e91d6ecba174376d. Report an issue: GitHub.