{"record":{"id":"eed84493cccce534","repo":"sgl-project/sglang","slug":"uspattention-masked-path-supports-ring-parallelism","errorCode":null,"errorMessage":"USPAttention masked path supports ring parallelism only for batch-1 tail-pad metadata on the FA backend.","messagePattern":"USPAttention masked path supports ring parallelism only for batch-1 tail-pad metadata on the FA backend\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/attention/layer.py","lineNumber":1053,"sourceCode":"                with sdpa_context:\n                    return torch.nn.functional.scaled_dot_product_attention(\n                        q_,\n                        k_,\n                        v_,\n                        attn_mask=mask,\n                        dropout_p=0.0,\n                        is_causal=False,\n                        scale=self.softmax_scale,\n                    ).transpose(1, 2)\n\n            if get_ring_parallel_world_size() > 1:\n                if (\n                    meta_only_pad\n                    and q.shape[0] == 1\n                    and self.backend == AttentionBackendEnum.FA\n                ):\n                    return self._forward_ring_tail_pad(q, k, v, attn_mask_meta)\n                raise NotImplementedError(\n                    \"USPAttention masked path supports ring parallelism only \"\n                    \"for batch-1 tail-pad metadata on the FA backend.\"\n                )\n            if attn_mask is not None and attn_mask.dim() != 2:\n                raise NotImplementedError(\n                    \"USPAttention masked SP path currently expects a [B, S_local] key mask.\"\n                )\n\n            sp_size = get_ulysses_parallel_world_size()\n            if sp_size > 1 and not qkv_pre_all_to_all:\n                qkv_fast = _ipc_input_a2a_qkv(q, k, v)\n                if qkv_fast is not None:\n                    q, k, v = qkv_fast\n                else:\n                    q, k, v = _usp_input_all_to_all_qkv(q, k, v)\n\n            if (\n                _VARLEN_FA_ENABLED","sourceCodeStart":1035,"sourceCodeEnd":1071,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/attention/layer.py#L1035-L1071","documentation":"Under ring parallelism, the USPAttention masked path only has a correct ring implementation for the narrow case of batch size 1, tail-padding-only mask metadata, on the FA backend. Any other masked configuration (batch > 1, non-tail-pad metadata, or a non-FA backend) has no ring-aware implementation and is rejected.","triggerScenarios":"Calling USPAttention.forward with an attn_mask/attn_mask_meta under ring parallel world size > 1 when NOT (meta_only_pad and q.shape[0] == 1 and backend == FA) — e.g. batch size > 1, a general 2D mask instead of pad metadata, or a non-FA backend.","commonSituations":"Batched video/image generation with padding masks under ring attention; switching the backend from FA to FlashInfer/Triton while ring parallelism is on; passing a boolean [B, S] mask instead of pad-only metadata.","solutions":["Restructure to batch size 1 with tail-padding and express the mask as pad-only attn_mask_meta on the FA backend","Or disable ring parallelism for masked batched workloads","Or switch to the unmasked/varlen path which has its own (non-ring) SP handling"],"exampleFix":"# before\nout = attn(q, k, v, attn_mask=bool_mask)  # ring parallel, batch=4\n# after\n# single request, tail pad metadata, FA backend\nout = attn(q, k, v[:1], k, v, attn_mask_meta=tail_pad_meta)","handlingStrategy":"validation","validationCode":"ring = get_ring_parallel_world_size() > 1\nif ring and attn_mask is not None:\n    supported = meta_only_pad and q.shape[0] == 1 and layer.backend == AttentionBackendEnum.FA\n    if not supported:\n        raise ValueError(\"ring masked path needs batch=1 tail-pad meta on FA\")","typeGuard":"def ring_masked_ok(q, meta_only_pad: bool, backend) -> bool:\n    return meta_only_pad and q.shape[0] == 1 and backend == AttentionBackendEnum.FA","tryCatchPattern":null,"preventionTips":["Serve one request per forward step with tail padding when using ring attention with masks","Pin the FA backend when ring parallelism is enabled","Prefer pad-only attn_mask_meta over general boolean masks under ring attention"],"tags":["attention","ring-parallelism","attention-mask","batch-size","fa-backend","not-implemented"],"backgroundTag":"parallelism-strategy-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}