{"record":{"id":"48cd67aad6c98715","repo":"sgl-project/sglang","slug":"max-seqlen-should-be-prepared-for-vision-flashinfe","errorCode":null,"errorMessage":"max_seqlen should be prepared for vision flashinfer_cudnn attention backend","messagePattern":"max_seqlen should be prepared for vision flashinfer_cudnn attention backend","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/vision.py","lineNumber":677,"sourceCode":"    ) -> torch.Tensor:\n        r\"\"\"\n        Args:\n            cu_seqlens: [b]\n        Returns:\n             [b * s, h, head_size]\n        \"\"\"\n        # ---- resolve sequence_lengths, packed indptrs, max_seqlen ----\n        if forward_metadata is not None and forward_metadata.packed_indptrs is not None:\n            sequence_lengths = forward_metadata.sequence_lengths\n            packed_cu_seqlens = forward_metadata.packed_indptrs\n            max_seqlen = forward_metadata.flashinfer_max_seqlen\n        else:\n            if \"sequence_lengths\" not in kwargs:\n                raise RuntimeError(\n                    \"sequence_lengths should be prepared for vision flashinfer_cudnn attention backend\"\n                )\n            if \"max_seqlen\" not in kwargs:\n                raise RuntimeError(\n                    \"max_seqlen should be prepared for vision flashinfer_cudnn attention backend\"\n                )\n            sequence_lengths = kwargs[\"sequence_lengths\"]\n            packed_cu_seqlens = cu_seqlens\n            max_seqlen = kwargs[\"max_seqlen\"]\n\n        # max_seqlen must be python int\n        if isinstance(max_seqlen, torch.Tensor):\n            if max_seqlen.is_cuda:\n                max_seqlen = int(max_seqlen.detach().cpu().item())\n            else:\n                max_seqlen = int(max_seqlen.item())\n        else:\n            max_seqlen = int(max_seqlen)\n\n        # flatten if caller gives (b, s, h, d)\n        is_reshaped = q.dim() == 4\n        if is_reshaped:","sourceCodeStart":659,"sourceCodeEnd":695,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/vision.py#L659-L695","documentation":"Companion check to the sequence_lengths guard: for flashinfer_cudnn vision attention without prepared metadata, the caller must also provide max_seqlen via kwargs. cudnn's varlen plan needs the max sequence length to size its kernels, and it cannot be inferred reliably from packed indptrs alone in this fallback path.","triggerScenarios":"forward_metadata is None (or packed_indptrs is None) and kwargs lacks 'max_seqlen' while using the flashinfer_cudnn vision backend — typically the caller passed sequence_lengths but not max_seqlen.","commonSituations":"Partially migrating a model to the flashinfer_cudnn contract (sequence_lengths added, max_seqlen forgotten); custom wrappers that compute lengths but skip the max; changes in SGLang that made max_seqlen an explicit requirement.","solutions":["Pass max_seqlen=int(sequence_lengths.max()) in the same kwargs as sequence_lengths.","Supply forward_metadata with packed_indptrs/flashinfer_max_seqlen so the prepared path is used instead.","Cache max_seqlen per batch shape when running under cuda-graph to keep it capture-stable."],"exampleFix":"# before\nout = attn(q, k, v, cu_seqlens=cu_seqlens, sequence_lengths=seq_lens)\n# after\nout = attn(q, k, v, cu_seqlens=cu_seqlens, sequence_lengths=seq_lens, max_seqlen=int(seq_lens.max()))","handlingStrategy":"validation","validationCode":"seq_lens = kwargs.get(\"sequence_lengths\") or (cu_seqlens[1:] - cu_seqlens[:-1])\nif \"max_seqlen\" not in kwargs:\n    kwargs[\"max_seqlen\"] = int(seq_lens.max().item())","typeGuard":"def has_cudnn_max_seqlen(metadata, kwargs) -> bool:\n    return (metadata is not None and getattr(metadata, \"flashinfer_max_seqlen\", None) is not None) or isinstance(kwargs.get(\"max_seqlen\"), int)","tryCatchPattern":null,"preventionTips":["Always pass sequence_lengths and max_seqlen as a pair for the cudnn vision backend.","Compute both in the same preprocessing step so they cannot drift.","Under cuda-graph, use a padded/static max_seqlen consistent across replays."],"tags":["sglang","vision-transformer","flashinfer-cudnn","kwargs-validation","max-seqlen"],"backgroundTag":"missing-required-argument","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}