vllm-project/vllm · error · ValueError

cuda_stream other than the current stream is not supported

Error message

cuda_stream other than the current stream is not supported

What it means

RayPPCommunicator dispatches ops on the actor's current CUDA stream only. If a cuda_stream argument is supplied and differs from current_stream(), the constructor raises ValueError, because the underlying vLLM PP communicator would issue NCCL ops on the wrong stream relative to the actor's execution stream.

Source

Thrown at vllm/distributed/device_communicators/ray_communicator.py:64

            world_size: The number of participating actors.
            comm_id: A unique communicator ID. This is just to conform with
                the Ray Communicator API and is not used.
            rank: The rank of this actor. If None, then the caller is not a
                participant of the RayPPCommunicator group (e.g., the Ray
                driver).
            actor_handles: A list of actor handles.
            cuda_stream: A CUDA stream to dispatch communication ops to. This
                is not supported.
            use_communication_streams: Whether to use communication streams.
                This is not supported.
        """
        self._world_size = world_size
        self._rank: int | None = None
        self._actor_handles = actor_handles
        if use_communication_streams:
            raise NotImplementedError("use_communication_streams is not supported")
        if cuda_stream is not None and cuda_stream != current_stream():
            raise ValueError(
                "cuda_stream other than the current stream is not supported"
            )

        if rank is not None:
            # Rank is not None, this is Ray worker
            assert ray.get_gpu_ids(), "RayPPCommunicator has no GPUs assigned"

            self._comm = get_pp_group().device_communicator
            assert self._comm is not None

            # Since we wrap around the vLLM _PP communicator, we use
            # the rank from the vLLM communicator, and ignore the rank
            # passed in from Ray.
            # TODO(rui): refactor the Ray Communicator API so that
            # it also supports no rank passed in.
            self._rank = self._comm.rank_in_group

            self._build_actor_rank_mapping()

View on GitHub (pinned to c794754062)

Solutions

Pass cuda_stream=None and let the communicator use current_stream()
Or pass exactly the current stream: cuda_stream=torch.cuda.current_stream()

Example fix

# before
comm = RayPPCommunicator(..., cuda_stream=torch.cuda.Stream())

# after
comm = RayPPCommunicator(..., cuda_stream=None)

Defensive patterns

Strategy: validation

Validate before calling

if cuda_stream is not None and cuda_stream != torch.cuda.current_stream():
    cuda_stream = None  # RayPPCommunicator only accepts current stream or None

Try / catch

try:
    RayPPCommunicator(..., cuda_stream=s)
except ValueError:
    s = None; RayPPCommunicator(..., cuda_stream=s)

Prevention

Pass cuda_stream=None unless the backend documents stream support
Never create ad-hoc streams for communicators you don't control
Feature-detect stream support per backend

When it happens

Trigger: Constructing RayPPCommunicator(..., cuda_stream=torch.cuda.Stream()) or any stream handle other than the current one captured at init.

Common situations: Generic code that always forwards a side stream for overlap; capturing a stream earlier and passing it after the actor migrated streams; migrating configs from a backend that accepted custom streams.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/ee30fd0cd297a21e. Report an issue: GitHub.