sgl-project/sglang · error · NotImplementedError

K/V-gather SP does not support video sparse attention.

Error message

K/V-gather SP does not support video sparse attention.

What it means

Video sparse attention (VSA) cannot run under the K/V-gather sequence-parallel attention mode, so the forward of the VSA attention layer refuses immediately when sp_attention_mode == 'kv_gather'.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/layer.py:551

    ) -> torch.Tensor:
        """Forward pass for distributed attention.

        Args:
            q (torch.Tensor): Query tensor [batch_size, seq_len, num_heads, head_dim]
            k (torch.Tensor): Key tensor [batch_size, seq_len, num_heads, head_dim]
            v (torch.Tensor): Value tensor [batch_size, seq_len, num_heads, head_dim]
            gate_compress (torch.Tensor): Gate compress tensor [batch_size, seq_len, num_heads, head_dim]
            replicated_q (Optional[torch.Tensor]): Replicated query tensor, typically for text tokens
            replicated_k (Optional[torch.Tensor]): Replicated key tensor
            replicated_v (Optional[torch.Tensor]): Replicated value tensor

        Returns:
            Tuple[torch.Tensor, Optional[torch.Tensor]]: A tuple containing:
                - o (torch.Tensor): Output tensor after attention for the main sequence
                - replicated_o (Optional[torch.Tensor]): Output tensor for replicated tokens, if provided
        """
        if self.sp_attention_mode == "kv_gather":
            raise NotImplementedError(
                "K/V-gather SP does not support video sparse attention."
            )
        # Check text tokens are not supported for VSA now
        assert (
            replicated_q is None and replicated_k is None and replicated_v is None
        ), "Replicated QKV is not supported for VSA now"
        # Check input shapes
        assert q.dim() == 4 and k.dim() == 4 and v.dim() == 4, "Expected 4D tensors"

        forward_context: ForwardContext = get_forward_context()
        ctx_attn_metadata = forward_context.attn_metadata

        # Stack QKV
        qkvg = torch.cat(
            [q, k, v, gate_compress], dim=0
        )  # [3, seq_len, num_heads, head_dim]

        # Redistribute heads across sequence dimension

View on GitHub (pinned to 0132848349)

Solutions

1. Set the SP attention mode to a supported value (not kv_gather) in the server/runtime configuration
2. Or disable sequence parallelism for video sparse attention workloads
3. Check for model-level guidance on which SP modes are validated for VSA and use those

Example fix

# before
server_args.sp_attention_mode = "kv_gather"  # video model with VSA
# after
server_args.sp_attention_mode = "a2a"  # or disable SP

if getattr(layer, "sp_attention_mode", None) == "kv_gather":
    raise ValueError("video sparse attention cannot run with kv_gather SP; change sp_attention_mode")

def vsa_sp_mode_supported(mode: str) -> bool:
    return mode != "kv_gather"

When it happens

Trigger: Running video sparse attention forward while the runtime config selected the kv_gather SP attention mode (e.g. launching a video model with --sp-attention-mode kv_gather or equivalent config).

Common situations: Enabling sequence parallelism with the kv_gather option for a video generation model that uses sparse attention; changing server args for throughput and hitting this unsupported combination; new deployment of a VSA model with SP defaults that pick kv_gather.

Related errors

• K/V-gather SP does not support varlen UlyssesAttention.
• UlyssesAttention's all-to-all spans the combined sequence pa
• Varlen USPAttention does not support ring parallelism yet.
• USPAttention's masked path does not support replicated prefi
• /v1/models ${response.status}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/163d585327228307. Report an issue: GitHub.