vllm-project/vllm · error · NotImplementedError

use_communication_streams is not supported

Error message

use_communication_streams is not supported

What it means

RayPPCommunicator implements the generic device-communicator interface on top of vLLM's pipeline-parallel group over Ray actors. It executes everything on the current CUDA stream, so the use_communication_streams flag (used by other backends to overlap comm and compute on separate streams) is explicitly rejected with NotImplementedError at construction time.

Source

Thrown at vllm/distributed/device_communicators/ray_communicator.py:62

        Args:
            world_size: The number of participating actors.
            comm_id: A unique communicator ID. This is just to conform with
                the Ray Communicator API and is not used.
            rank: The rank of this actor. If None, then the caller is not a
                participant of the RayPPCommunicator group (e.g., the Ray
                driver).
            actor_handles: A list of actor handles.
            cuda_stream: A CUDA stream to dispatch communication ops to. This
                is not supported.
            use_communication_streams: Whether to use communication streams.
                This is not supported.
        """
        self._world_size = world_size
        self._rank: int | None = None
        self._actor_handles = actor_handles
        if use_communication_streams:
            raise NotImplementedError("use_communication_streams is not supported")
        if cuda_stream is not None and cuda_stream != current_stream():
            raise ValueError(
                "cuda_stream other than the current stream is not supported"
            )

        if rank is not None:
            # Rank is not None, this is Ray worker
            assert ray.get_gpu_ids(), "RayPPCommunicator has no GPUs assigned"

            self._comm = get_pp_group().device_communicator
            assert self._comm is not None

            # Since we wrap around the vLLM _PP communicator, we use
            # the rank from the vLLM communicator, and ignore the rank
            # passed in from Ray.
            # TODO(rui): refactor the Ray Communicator API so that
            # it also supports no rank passed in.
            self._rank = self._comm.rank_in_group

View on GitHub (pinned to c794754062)

Solutions

  1. Pass use_communication_streams=False (or omit it) when constructing RayPPCommunicator
  2. If overlap is required, use a non-Ray communicator backend that supports communication streams

Example fix

# before
comm = RayPPCommunicator(world_size, rank, actor_handles, use_communication_streams=True)

# after
comm = RayPPCommunicator(world_size, rank, actor_handles)  # default False
Defensive patterns

Strategy: validation

Validate before calling

if isinstance(comm_cls, type) and comm_cls.__name__ == "RayPPCommunicator":
    kwargs.pop("use_communication_streams", None)  # backend only supports current-stream execution

Try / catch

try:
    RayPPCommunicator(..., use_communication_streams=flag)
except NotImplementedError:
    flag = False  # retry without comm streams

Prevention

When it happens

Trigger: Instantiating RayPPCommunicator(..., use_communication_streams=True), typically from generic launcher code that sets the flag based on --enable-symm-mem / comm-overlap options regardless of backend.

Common situations: Turning on communication/compute overlap in a Ray-based pipeline-parallel deployment; sharing config between a NCCL-based communicator and the Ray backend.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/541f730a4b0b20c6. Report an issue: GitHub.