{"record":{"id":"163d585327228307","repo":"sgl-project/sglang","slug":"k-v-gather-sp-does-not-support-video-sparse-attent","errorCode":null,"errorMessage":"K/V-gather SP does not support video sparse attention.","messagePattern":"K/V-gather SP does not support video sparse attention\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/attention/layer.py","lineNumber":551,"sourceCode":"    ) -> torch.Tensor:\n        \"\"\"Forward pass for distributed attention.\n\n        Args:\n            q (torch.Tensor): Query tensor [batch_size, seq_len, num_heads, head_dim]\n            k (torch.Tensor): Key tensor [batch_size, seq_len, num_heads, head_dim]\n            v (torch.Tensor): Value tensor [batch_size, seq_len, num_heads, head_dim]\n            gate_compress (torch.Tensor): Gate compress tensor [batch_size, seq_len, num_heads, head_dim]\n            replicated_q (Optional[torch.Tensor]): Replicated query tensor, typically for text tokens\n            replicated_k (Optional[torch.Tensor]): Replicated key tensor\n            replicated_v (Optional[torch.Tensor]): Replicated value tensor\n\n        Returns:\n            Tuple[torch.Tensor, Optional[torch.Tensor]]: A tuple containing:\n                - o (torch.Tensor): Output tensor after attention for the main sequence\n                - replicated_o (Optional[torch.Tensor]): Output tensor for replicated tokens, if provided\n        \"\"\"\n        if self.sp_attention_mode == \"kv_gather\":\n            raise NotImplementedError(\n                \"K/V-gather SP does not support video sparse attention.\"\n            )\n        # Check text tokens are not supported for VSA now\n        assert (\n            replicated_q is None and replicated_k is None and replicated_v is None\n        ), \"Replicated QKV is not supported for VSA now\"\n        # Check input shapes\n        assert q.dim() == 4 and k.dim() == 4 and v.dim() == 4, \"Expected 4D tensors\"\n\n        forward_context: ForwardContext = get_forward_context()\n        ctx_attn_metadata = forward_context.attn_metadata\n\n        # Stack QKV\n        qkvg = torch.cat(\n            [q, k, v, gate_compress], dim=0\n        )  # [3, seq_len, num_heads, head_dim]\n\n        # Redistribute heads across sequence dimension","sourceCodeStart":533,"sourceCodeEnd":569,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/attention/layer.py#L533-L569","documentation":"Video sparse attention (VSA) cannot run under the K/V-gather sequence-parallel attention mode, so the forward of the VSA attention layer refuses immediately when sp_attention_mode == 'kv_gather'.","triggerScenarios":"Running video sparse attention forward while the runtime config selected the kv_gather SP attention mode (e.g. launching a video model with --sp-attention-mode kv_gather or equivalent config).","commonSituations":"Enabling sequence parallelism with the kv_gather option for a video generation model that uses sparse attention; changing server args for throughput and hitting this unsupported combination; new deployment of a VSA model with SP defaults that pick kv_gather.","solutions":["Set the SP attention mode to a supported value (not kv_gather) in the server/runtime configuration","Or disable sequence parallelism for video sparse attention workloads","Check for model-level guidance on which SP modes are validated for VSA and use those"],"exampleFix":"# before\nserver_args.sp_attention_mode = \"kv_gather\"  # video model with VSA\n# after\nserver_args.sp_attention_mode = \"a2a\"  # or disable SP","handlingStrategy":"validation","validationCode":"if getattr(layer, \"sp_attention_mode\", None) == \"kv_gather\":\n    raise ValueError(\"video sparse attention cannot run with kv_gather SP; change sp_attention_mode\")","typeGuard":"def vsa_sp_mode_supported(mode: str) -> bool:\n    return mode != \"kv_gather\"","tryCatchPattern":null,"preventionTips":["Validate the (model, sp_attention_mode) pair at server startup","Keep a matrix of supported SP modes per attention type in config checks"],"tags":["video-sparse-attention","sequence-parallel","kv-gather","not-implemented"],"backgroundTag":"parallelism-strategy-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}