sgl-project/sglang · error · NotImplementedError

Lightning (seg_la) linear-attention backend does not support

Error message

Lightning (seg_la) linear-attention backend does not support speculative decoding with topk > 1 (got topk={self.topk}); seg_la verifies a draft tree as a chain. Use --speculative-eagle-topk 1.

What it means

The Lightning (seg_la) linear-attention backend verifies EAGLE draft tokens as a chain and has no parent-indices plumbing for tree drafts; with topk > 1 it would silently commit wrong mamba states, so __init__ fails fast with NotImplementedError and suggests --speculative-eagle-topk 1.

Source

Thrown at python/sglang/srt/layers/attention/linear/lightning_backend.py:49

    - If spec decoding
        - FlashAttentionBackend will be init once for the target worker
        - FlashAttentionMultiStepBackend will be once for the draft worker
            - It will spawn num_steps FlashAttentionBackend for the draft worker

    Note about CUDA Graph:
    - We only support CUDA Graph for Decode (Normal Decode and Draft Decode) and Target Verify.
    - We don't support CUDA Graph for Extend and Draft Extend.
    - When server init, init_cuda_graph_state will be called first and then init_cuda_graph_capture will be called.
    - For each forward batch, init_replay_cuda_graph will be called first and then replay the graph.
    """

    def __init__(self, model_runner: ModelRunner):
        super().__init__(model_runner)
        # seg_la processes draft tokens as a chain -- it has no parent-indices
        # plumbing for tree-shaped drafts, so spec v2 tree verify (topk > 1) would
        # commit wrong mamba states silently. Fail fast instead of mis-decoding.
        if self.topk > 1:
            raise NotImplementedError(
                "Lightning (seg_la) linear-attention backend does not support "
                f"speculative decoding with topk > 1 (got topk={self.topk}); "
                "seg_la verifies a draft tree as a chain. Use "
                "--speculative-eagle-topk 1."
            )
        # lightning attn does not need conv cache, but to keep the interface for mamba cache
        self.conv_states_shape = (
            model_runner.req_to_token_pool.mamba_pool.mamba_cache.conv[0].shape
        )

        assert not (
            model_runner.sliding_window_size is not None
            and model_runner.model_config.is_encoder_decoder
        ), "Sliding window and cross attention are not supported together"

        # extra metadata for handling speculative decoding topk > 1, extended draft decode and verify
        self.max_context_len = model_runner.model_config.context_len
        self.device = model_runner.device

View on GitHub (pinned to 0132848349)

Solutions

  1. Set --speculative-eagle-topk 1
  2. Use a linear-attention backend that supports tree verify if topk>1 is required
  3. Disable speculative decoding for this model

Example fix

# before
--speculative-eagle-topk 4 --linear-attn-backend lightning
# after
--speculative-eagle-topk 1 --linear-attn-backend lightning
Defensive patterns

Strategy: validation

Validate before calling

topk = server_args.speculative_eagle_topk or 1
if linear_attn_backend == 'lightning' and topk > 1:
    raise SystemExit('seg_la requires --speculative-eagle-topk 1; got %d' % topk)

Type guard

null

Prevention

When it happens

Trigger: Constructing the Lightning linear-attention backend while ModelRunner's speculative config has topk > 1 (e.g. --speculative-eagle-topk 4 on a hybrid linear-attention model).

Common situations: Copying an EAGLE3 topk>1 config onto a Mamba/GDN/KDA hybrid model served with the seg_la backend; upgrading spec config without checking backend constraints.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a8fd84aa8fe31065. Report an issue: GitHub.