sgl-project/sglang · error · RuntimeError

max_seqlen should be prepared for vision flashinfer_cudnn…

Error message

max_seqlen should be prepared for vision flashinfer_cudnn attention backend

What it means

Companion check to the sequence_lengths guard: for flashinfer_cudnn vision attention without prepared metadata, the caller must also provide max_seqlen via kwargs. cudnn's varlen plan needs the max sequence length to size its kernels, and it cannot be inferred reliably from packed indptrs alone in this fallback path.

Solutions

Pass max_seqlen=int(sequence_lengths.max()) in the same kwargs as sequence_lengths.
Supply forward_metadata with packed_indptrs/flashinfer_max_seqlen so the prepared path is used instead.
Cache max_seqlen per batch shape when running under cuda-graph to keep it capture-stable.

Example fix

# before
out = attn(q, k, v, cu_seqlens=cu_seqlens, sequence_lengths=seq_lens)
# after
out = attn(q, k, v, cu_seqlens=cu_seqlens, sequence_lengths=seq_lens, max_seqlen=int(seq_lens.max()))

Defensive patterns

Strategy: validation

Validate before calling

seq_lens = kwargs.get("sequence_lengths") or (cu_seqlens[1:] - cu_seqlens[:-1])
if "max_seqlen" not in kwargs:
    kwargs["max_seqlen"] = int(seq_lens.max().item())

Type guard

def has_cudnn_max_seqlen(metadata, kwargs) -> bool:
    return (metadata is not None and getattr(metadata, "flashinfer_max_seqlen", None) is not None) or isinstance(kwargs.get("max_seqlen"), int)

Prevention

Always pass sequence_lengths and max_seqlen as a pair for the cudnn vision backend.
Compute both in the same preprocessing step so they cannot drift.
Under cuda-graph, use a padded/static max_seqlen consistent across replays.

When it happens

Trigger: forward_metadata is None (or packed_indptrs is None) and kwargs lacks 'max_seqlen' while using the flashinfer_cudnn vision backend — typically the caller passed sequence_lengths but not max_seqlen.

Common situations: Partially migrating a model to the flashinfer_cudnn contract (sequence_lengths added, max_seqlen forgotten); custom wrappers that compute lengths but skip the max; changes in SGLang that made max_seqlen an explicit requirement.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/48cd67aad6c98715. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/layers/attention/vision.py:677

    ) -> torch.Tensor:
        r"""
        Args:
            cu_seqlens: [b]
        Returns:
             [b * s, h, head_size]
        """
        # ---- resolve sequence_lengths, packed indptrs, max_seqlen ----
        if forward_metadata is not None and forward_metadata.packed_indptrs is not None:
            sequence_lengths = forward_metadata.sequence_lengths
            packed_cu_seqlens = forward_metadata.packed_indptrs
            max_seqlen = forward_metadata.flashinfer_max_seqlen
        else:
            if "sequence_lengths" not in kwargs:
                raise RuntimeError(
                    "sequence_lengths should be prepared for vision flashinfer_cudnn attention backend"
                )
            if "max_seqlen" not in kwargs:
                raise RuntimeError(
                    "max_seqlen should be prepared for vision flashinfer_cudnn attention backend"
                )
            sequence_lengths = kwargs["sequence_lengths"]
            packed_cu_seqlens = cu_seqlens
            max_seqlen = kwargs["max_seqlen"]

        # max_seqlen must be python int
        if isinstance(max_seqlen, torch.Tensor):
            if max_seqlen.is_cuda:
                max_seqlen = int(max_seqlen.detach().cpu().item())
            else:
                max_seqlen = int(max_seqlen.item())
        else:
            max_seqlen = int(max_seqlen)

        # flatten if caller gives (b, s, h, d)
        is_reshaped = q.dim() == 4
        if is_reshaped:

View on GitHub (pinned to 0132848349)