{"record":{"id":"dcf993003f2c967c","repo":"sgl-project/sglang","slug":"sequence-lengths-should-be-prepared-for-vision-fla","errorCode":null,"errorMessage":"sequence_lengths should be prepared for vision flashinfer_cudnn attention backend","messagePattern":"sequence_lengths should be prepared for vision flashinfer_cudnn attention backend","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/vision.py","lineNumber":673,"sourceCode":"        seq_len: int,\n        softmax_scale: Optional[float] = None,\n        forward_metadata: Optional[VisionAttentionMetadata] = None,\n        **kwargs,\n    ) -> torch.Tensor:\n        r\"\"\"\n        Args:\n            cu_seqlens: [b]\n        Returns:\n             [b * s, h, head_size]\n        \"\"\"\n        # ---- resolve sequence_lengths, packed indptrs, max_seqlen ----\n        if forward_metadata is not None and forward_metadata.packed_indptrs is not None:\n            sequence_lengths = forward_metadata.sequence_lengths\n            packed_cu_seqlens = forward_metadata.packed_indptrs\n            max_seqlen = forward_metadata.flashinfer_max_seqlen\n        else:\n            if \"sequence_lengths\" not in kwargs:\n                raise RuntimeError(\n                    \"sequence_lengths should be prepared for vision flashinfer_cudnn attention backend\"\n                )\n            if \"max_seqlen\" not in kwargs:\n                raise RuntimeError(\n                    \"max_seqlen should be prepared for vision flashinfer_cudnn attention backend\"\n                )\n            sequence_lengths = kwargs[\"sequence_lengths\"]\n            packed_cu_seqlens = cu_seqlens\n            max_seqlen = kwargs[\"max_seqlen\"]\n\n        # max_seqlen must be python int\n        if isinstance(max_seqlen, torch.Tensor):\n            if max_seqlen.is_cuda:\n                max_seqlen = int(max_seqlen.detach().cpu().item())\n            else:\n                max_seqlen = int(max_seqlen.item())\n        else:\n            max_seqlen = int(max_seqlen)","sourceCodeStart":655,"sourceCodeEnd":691,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/vision.py#L655-L691","documentation":"For the flashinfer_cudnn vision attention backend, when no prepared forward_metadata (with packed_indptrs) is supplied, the caller must pass sequence_lengths (a (B,) tensor of per-image sequence lengths) via kwargs. The error fires when neither metadata nor kwargs contain it, because cudnn prefill needs the per-batch lengths to run its varlen wrapper.","triggerScenarios":"Calling the flashinfer_cudnn ViT attention forward without forward_metadata (or with metadata whose packed_indptrs is None) and without kwargs['sequence_lengths'] — e.g. a custom model forward that only passes q/k/v and cu_seqlens.","commonSituations":"Adding flashinfer_cudnn as the vision attention backend to a new multimodal model without updating its forward call; running outside SGLang's runner (which normally prepares forward_metadata) so the fallback kwargs path is hit; version upgrades that introduced the kwargs contract.","solutions":["Pass sequence_lengths (and max_seqlen) in kwargs: attn(q, k, v, cu_seqlens=..., sequence_lengths=..., max_seqlen=...).","Prefer supplying the backend-prepared forward_metadata so the packed indptrs path is used.","Compute sequence_lengths as diffs of cu_seqlens if you only have the cumulative form."],"exampleFix":"# before\nout = attn(q, k, v, cu_seqlens=cu_seqlens)\n# after\nseq_lens = cu_seqlens[1:] - cu_seqlens[:-1]\nout = attn(q, k, v, cu_seqlens=cu_seqlens, sequence_lengths=seq_lens, max_seqlen=int(seq_lens.max()))","handlingStrategy":"validation","validationCode":"if metadata is None or metadata.packed_indptrs is None:\n    if \"sequence_lengths\" not in kwargs:\n        seq_lens = cu_seqlens[1:] - cu_seqlens[:-1]\n        kwargs[\"sequence_lengths\"] = seq_lens.to(dtype=torch.int32, device=q.device)","typeGuard":"def has_cudnn_seq_lens(metadata, kwargs) -> bool:\n    return (metadata is not None and getattr(metadata, \"packed_indptrs\", None) is not None) or isinstance(kwargs.get(\"sequence_lengths\"), torch.Tensor)","tryCatchPattern":null,"preventionTips":["When wiring a new model to flashinfer_cudnn, pass both sequence_lengths and max_seqlen explicitly.","Derive sequence_lengths from cu_seqlens diffs once in a preprocessing helper.","Write an integration test that runs the model without prepared metadata to exercise the kwargs path."],"tags":["sglang","vision-transformer","flashinfer-cudnn","kwargs-validation","multimodal"],"backgroundTag":"missing-required-argument","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}