{"record":{"id":"ee30fd0cd297a21e","repo":"vllm-project/vllm","slug":"cuda-stream-other-than-the-current-stream-is-not-s","errorCode":null,"errorMessage":"cuda_stream other than the current stream is not supported","messagePattern":"cuda_stream other than the current stream is not supported","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/device_communicators/ray_communicator.py","lineNumber":64,"sourceCode":"            world_size: The number of participating actors.\n            comm_id: A unique communicator ID. This is just to conform with\n                the Ray Communicator API and is not used.\n            rank: The rank of this actor. If None, then the caller is not a\n                participant of the RayPPCommunicator group (e.g., the Ray\n                driver).\n            actor_handles: A list of actor handles.\n            cuda_stream: A CUDA stream to dispatch communication ops to. This\n                is not supported.\n            use_communication_streams: Whether to use communication streams.\n                This is not supported.\n        \"\"\"\n        self._world_size = world_size\n        self._rank: int | None = None\n        self._actor_handles = actor_handles\n        if use_communication_streams:\n            raise NotImplementedError(\"use_communication_streams is not supported\")\n        if cuda_stream is not None and cuda_stream != current_stream():\n            raise ValueError(\n                \"cuda_stream other than the current stream is not supported\"\n            )\n\n        if rank is not None:\n            # Rank is not None, this is Ray worker\n            assert ray.get_gpu_ids(), \"RayPPCommunicator has no GPUs assigned\"\n\n            self._comm = get_pp_group().device_communicator\n            assert self._comm is not None\n\n            # Since we wrap around the vLLM _PP communicator, we use\n            # the rank from the vLLM communicator, and ignore the rank\n            # passed in from Ray.\n            # TODO(rui): refactor the Ray Communicator API so that\n            # it also supports no rank passed in.\n            self._rank = self._comm.rank_in_group\n\n            self._build_actor_rank_mapping()","sourceCodeStart":46,"sourceCodeEnd":82,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/device_communicators/ray_communicator.py#L46-L82","documentation":"RayPPCommunicator dispatches ops on the actor's current CUDA stream only. If a cuda_stream argument is supplied and differs from current_stream(), the constructor raises ValueError, because the underlying vLLM PP communicator would issue NCCL ops on the wrong stream relative to the actor's execution stream.","triggerScenarios":"Constructing RayPPCommunicator(..., cuda_stream=torch.cuda.Stream()) or any stream handle other than the current one captured at init.","commonSituations":"Generic code that always forwards a side stream for overlap; capturing a stream earlier and passing it after the actor migrated streams; migrating configs from a backend that accepted custom streams.","solutions":["Pass cuda_stream=None and let the communicator use current_stream()","Or pass exactly the current stream: cuda_stream=torch.cuda.current_stream()"],"exampleFix":"# before\ncomm = RayPPCommunicator(..., cuda_stream=torch.cuda.Stream())\n\n# after\ncomm = RayPPCommunicator(..., cuda_stream=None)","handlingStrategy":"validation","validationCode":"if cuda_stream is not None and cuda_stream != torch.cuda.current_stream():\n    cuda_stream = None  # RayPPCommunicator only accepts current stream or None","typeGuard":null,"tryCatchPattern":"try:\n    RayPPCommunicator(..., cuda_stream=s)\nexcept ValueError:\n    s = None; RayPPCommunicator(..., cuda_stream=s)","preventionTips":["Pass cuda_stream=None unless the backend documents stream support","Never create ad-hoc streams for communicators you don't control","Feature-detect stream support per backend"],"tags":["ray","cuda-stream","pipeline-parallel","distributed"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}