{"record":{"id":"354d2c9af887182f","repo":"sgl-project/sglang","slug":"varlen-uspattention-does-not-support-ring-parallel","errorCode":null,"errorMessage":"Varlen USPAttention does not support ring parallelism yet.","messagePattern":"Varlen USPAttention does not support ring parallelism yet\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/attention/layer.py","lineNumber":878,"sourceCode":"        ctx_attn_metadata = forward_context.attn_metadata\n        effective_skip_sp = (\n            self.skip_sequence_parallel or skip_sequence_parallel_override\n        )\n        if seq_lens is not None:\n            assert (\n                attn_mask is None\n                and attn_mask_meta is None\n                and not num_replicated_prefix\n                and not num_replicated_suffix\n                and not num_replicated_kv_prefix\n            ), \"Varlen USPAttention does not support masks or replicated tokens\"\n            if effective_skip_sp or get_sequence_parallel_world_size() == 1:\n                return self.attn_impl.forward(q, k, v, ctx_attn_metadata)\n            if get_ring_parallel_world_size() > 1:\n                # The varlen all-to-all spans the combined SP group and is not\n                # ring-aware; it would shuffle rows across ring ranks instead\n                # of rotating KV, corrupting the output silently.\n                raise NotImplementedError(\n                    \"Varlen USPAttention does not support ring parallelism yet.\"\n                )\n            qkv = torch.cat([q, k, v], dim=0)\n            qkv = _usp_input_all_to_all_varlen(qkv, seq_lens, head_dim=2)\n            qkv = self.attn_impl.preprocess_qkv(qkv, ctx_attn_metadata)\n            q, k, v = qkv.chunk(3, dim=0)\n            out = self.attn_impl.forward(q, k, v, ctx_attn_metadata)\n            out = self.attn_impl.postprocess_output(out, ctx_attn_metadata)\n            return _usp_output_all_to_all_varlen(out, seq_lens, head_dim=2)\n\n        if isinstance(attn_mask_meta, DynamicVarlenMaskMeta):\n            attn_mask_meta = attn_mask_meta.resolve(attn_mask)\n\n        # Tail-pad meta alone (sp_shard.tail_attn_meta; mask derivable from the\n        # pad span) also opts into the masked SP branch. gap_* = legacy alias.\n        meta_pad_start = meta_pad_end = None\n        if attn_mask_meta is not None:\n            meta_pad_start = attn_mask_meta.get(","sourceCodeStart":860,"sourceCodeEnd":896,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/attention/layer.py#L860-L896","documentation":"The varlen path of USPAttention performs an all-to-all over the combined sequence-parallel group; under ring parallelism (ring world size > 1) this would shuffle rows across ring ranks instead of rotating KV, silently corrupting output, so it raises NotImplementedError instead.","triggerScenarios":"Calling USPAttention.forward with seq_lens not None (ragged/varlen batch), sequence parallel world size > 1, effective_skip_sp False, and ring parallel world size > 1.","commonSituations":"Serving variable-length multimodal prompts on a ring-attention-enabled deployment; enabling ring parallelism for throughput on a workload that produces varlen batches; a config change that turns on ring attention for a model whose batches are ragged.","solutions":["Disable ring parallelism (run with ring world size 1) for varlen workloads","Or pad/bucket requests to uniform lengths so the non-varlen ring path is used","Track the upstream task implementing varlen ring support and upgrade once available"],"exampleFix":"# before\n# launch with ring parallelism, batch has variable lengths\nout = attn(q, k, v, seq_lens=lens)\n# after\n# disable ring parallelism in server args, keep SP + varlen\nout = attn(q, k, v, seq_lens=lens)","handlingStrategy":"validation","validationCode":"if seq_lens is not None and get_ring_parallel_world_size() > 1 and get_sequence_parallel_world_size() > 1:\n    raise ValueError(\"varlen + ring parallelism unsupported; disable ring or pad batch\")","typeGuard":"def varlen_ring_ok(seq_lens, ring_ws: int) -> bool:\n    return seq_lens is None or ring_ws == 1","tryCatchPattern":null,"preventionTips":["Pad or bucket requests to uniform lengths when ring attention is enabled","Disable ring parallelism for ragged multimodal serving traffic"],"tags":["attention","varlen","ring-parallelism","sequence-parallel","not-implemented"],"backgroundTag":"varlen-batch-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}