sgl-project/sglang · critical · RuntimeError

SGLANG_DSA_TOPK_BROADCAST requires PyNCCL during CUDA graph

Error message

SGLANG_DSA_TOPK_BROADCAST requires PyNCCL during CUDA graph capture.

What it means

When SGLANG_DSA_TOPK_BROADCAST broadcasts DSA top-k indices from rank 0 during CUDA graph capture, it must use PyNCCL (the graph-capturable NCCL path). If the attention-TP group has no pynccl_comm, broadcasting inside a captured graph is impossible and the indexer raises.

Source

Thrown at python/sglang/srt/layers/attention/dsa/dsa_indexer.py:159

    from sglang.srt.layers.attention.dsa.dsa_prefill_cuda_graph import (
        logits_head_gate_graph,
        scale_head_gate_graph,
    )

    @register_custom_op(mutates_args=["topk_indices"])
    @register_split_op()
    def broadcast_indexer_topk_from_rank0_(topk_indices: torch.Tensor) -> None:
        _broadcast_indexer_topk_from_rank0_impl(topk_indices)


def _broadcast_indexer_topk_from_rank0_impl(topk_indices: torch.Tensor) -> None:
    group = get_attn_tp_group()
    if group.world_size == 1:
        return

    if topk_indices.device.type == "cuda" and torch.cuda.is_current_stream_capturing():
        if group.pynccl_comm is None:
            raise RuntimeError(
                "SGLANG_DSA_TOPK_BROADCAST requires PyNCCL during CUDA graph capture."
            )
        with group.pynccl_comm.change_state(enable=True):
            group.pynccl_comm.broadcast(topk_indices, src=0)
    else:
        group.broadcast(topk_indices, src=0)


def _broadcast_indexer_topk_from_rank0(
    topk_indices: Optional[torch.Tensor],
) -> Optional[torch.Tensor]:
    # Sync only the finalized indexer output. Internal topk_transform calls can
    # be chunked differently across ranks, which would make collectives diverge.
    if topk_indices is None or not envs.SGLANG_DSA_TOPK_BROADCAST.get():
        return topk_indices

    if is_in_tc_piecewise_cuda_graph():
        broadcast_indexer_topk_from_rank0_(topk_indices)

View on GitHub (pinned to 0132848349)

Solutions

Ensure PyNCCL is enabled/initialized before CUDA graph capture (default init path; avoid flags that disable it)
Initialize/attach the pynccl communicator on the attn TP group before capture (e.g. run one eager warmup forward so communicators are created)
Disable SGLANG_DSA_TOPK_BROADCAST if you don't need rank-0 topk broadcast

Defensive patterns

Strategy: validation

Validate before calling

from sglang.srt.distributed import get_attn_tp_group
g = get_attn_tp_group()
if g.world_size > 1 and g.pynccl_comm is None and os.environ.get("SGLANG_DSA_TOPK_BROADCAST"):
    # run one eager step to init communicators before capture
    model_runner.warmup_eager()

Prevention

Always run an eager warmup forward before CUDA graph capture so NCCL/PyNCCL communicators exist
Avoid flags that disable NCCL communicator initialization when using DSA broadcast on TP > 1

When it happens

Trigger: _broadcast_indexer_topk_from_rank0_impl running with attn TP world_size > 1, topk_indices on CUDA, torch.cuda.is_current_stream_capturing() true, and get_attn_tp_group().pynccl_comm is None — i.e. graph capture started without the PyNCCL communicator initialized.

Common situations: Disabling the custom allreduce/NCCL path (--disable-custom-all-reduce plus missing pynccl init), startup ordering regressions where graph capture happens before pynccl_comm setup, or flags that skip NCCL communicator creation on single-node TP runs with DSA models.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/192f6f7df804f40d. Report an issue: GitHub.