sgl-project/sglang · error · RuntimeError

sequence_lengths should be prepared for vision flashinfer_cu

Error message

sequence_lengths should be prepared for vision flashinfer_cudnn attention backend

What it means

For the flashinfer_cudnn vision attention backend, when no prepared forward_metadata (with packed_indptrs) is supplied, the caller must pass sequence_lengths (a (B,) tensor of per-image sequence lengths) via kwargs. The error fires when neither metadata nor kwargs contain it, because cudnn prefill needs the per-batch lengths to run its varlen wrapper.

Source

Thrown at python/sglang/srt/layers/attention/vision.py:673

        seq_len: int,
        softmax_scale: Optional[float] = None,
        forward_metadata: Optional[VisionAttentionMetadata] = None,
        **kwargs,
    ) -> torch.Tensor:
        r"""
        Args:
            cu_seqlens: [b]
        Returns:
             [b * s, h, head_size]
        """
        # ---- resolve sequence_lengths, packed indptrs, max_seqlen ----
        if forward_metadata is not None and forward_metadata.packed_indptrs is not None:
            sequence_lengths = forward_metadata.sequence_lengths
            packed_cu_seqlens = forward_metadata.packed_indptrs
            max_seqlen = forward_metadata.flashinfer_max_seqlen
        else:
            if "sequence_lengths" not in kwargs:
                raise RuntimeError(
                    "sequence_lengths should be prepared for vision flashinfer_cudnn attention backend"
                )
            if "max_seqlen" not in kwargs:
                raise RuntimeError(
                    "max_seqlen should be prepared for vision flashinfer_cudnn attention backend"
                )
            sequence_lengths = kwargs["sequence_lengths"]
            packed_cu_seqlens = cu_seqlens
            max_seqlen = kwargs["max_seqlen"]

        # max_seqlen must be python int
        if isinstance(max_seqlen, torch.Tensor):
            if max_seqlen.is_cuda:
                max_seqlen = int(max_seqlen.detach().cpu().item())
            else:
                max_seqlen = int(max_seqlen.item())
        else:
            max_seqlen = int(max_seqlen)

View on GitHub (pinned to 0132848349)

Solutions

1. Pass sequence_lengths (and max_seqlen) in kwargs: attn(q, k, v, cu_seqlens=..., sequence_lengths=..., max_seqlen=...).
2. Prefer supplying the backend-prepared forward_metadata so the packed indptrs path is used.
3. Compute sequence_lengths as diffs of cu_seqlens if you only have the cumulative form.

Example fix

# before
out = attn(q, k, v, cu_seqlens=cu_seqlens)
# after
seq_lens = cu_seqlens[1:] - cu_seqlens[:-1]
out = attn(q, k, v, cu_seqlens=cu_seqlens, sequence_lengths=seq_lens, max_seqlen=int(seq_lens.max()))

if metadata is None or metadata.packed_indptrs is None:
    if "sequence_lengths" not in kwargs:
        seq_lens = cu_seqlens[1:] - cu_seqlens[:-1]
        kwargs["sequence_lengths"] = seq_lens.to(dtype=torch.int32, device=q.device)

def has_cudnn_seq_lens(metadata, kwargs) -> bool:
    return (metadata is not None and getattr(metadata, "packed_indptrs", None) is not None) or isinstance(kwargs.get("sequence_lengths"), torch.Tensor)

When it happens

Trigger: Calling the flashinfer_cudnn ViT attention forward without forward_metadata (or with metadata whose packed_indptrs is None) and without kwargs['sequence_lengths'] — e.g. a custom model forward that only passes q/k/v and cu_seqlens.

Common situations: Adding flashinfer_cudnn as the vision attention backend to a new multimodal model without updating its forward call; running outside SGLang's runner (which normally prepares forward_metadata) so the fallback kwargs path is hit; version upgrades that introduced the kwargs contract.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• output_ws should be prepared for cuda-graph mode
• max_seqlen should be prepared for vision flashinfer_cudnn at
• flashinfer_cudnn expects packed indptrs as a torch.Tensor
• mask_search_files_path_pos, mask_search_files_path_neg, and
• {selection_error}{component_suffix}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/dcf993003f2c967c. Report an issue: GitHub.