{"record":{"id":"ffbed2bb70a79900","repo":"sgl-project/sglang","slug":"self-transport-name-requires-a-cuda-tensor","errorCode":null,"errorMessage":"{self.transport_name} requires a CUDA tensor","messagePattern":"(.+?) requires a CUDA tensor","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/multimodal/transport/memory_pool.py","lineNumber":335,"sourceCode":"    def _recycle_loop(self) -> None:\n        torch.cuda.set_device(self.device_id)\n        while not self._recycler_stop_event.is_set():\n            try:\n                with self._lock, torch.cuda.device(self.device_id):\n                    self._recycle_ready_leases_locked()\n            except Exception:\n                logger.warning(\n                    \"%s multimodal pool recycle failed\",\n                    self.transport_name,\n                    exc_info=True,\n                )\n            self._recycler_stop_event.wait(self._recycle_interval)\n\n    def copy_tensor(\n        self, tensor: torch.Tensor\n    ) -> tuple[Optional[PoolLease], Optional[torch.Tensor]]:\n        if not tensor.is_cuda:\n            raise ValueError(f\"{self.transport_name} requires a CUDA tensor\")\n        source = tensor.contiguous()\n        nbytes = source.numel() * source.element_size()\n        if nbytes == 0:\n            raise ValueError(f\"{self.transport_name} cannot transport an empty tensor\")\n        with self._lock:\n            lease = self._allocate_locked(nbytes)\n        if lease is None:\n            return None, None\n\n        try:\n            with torch.cuda.device(self.device_id):\n                destination = self.byte_tensor[lease.start : lease.start + lease.nbytes]\n                destination.copy_(\n                    source.view(torch.uint8).reshape(-1), non_blocking=True\n                )\n                stream_write_value32(\n                    self.device_id,\n                    self.base_address + lease.ready_byte_offset,","sourceCodeStart":317,"sourceCodeEnd":353,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/multimodal/transport/memory_pool.py#L317-L353","documentation":"copy_tensor performs a device-to-device copy into the shared pool, so the source tensor must already be on CUDA. A CPU tensor would trigger an implicit slow H2D copy or crash, so the pool rejects it upfront.","triggerScenarios":"Passing a tensor loaded on CPU (e.g. from preprocessing done on host) to wrap_tensor/copy_tensor; using .to('cpu') for debugging and forgetting to move it back.","commonSituations":"Multimodal preprocessing pipelines that produce CPU tensors (PIL/numpy origins); mixed-device code across ranks.","solutions":["Call tensor = tensor.to(device, non_blocking=True) (with a sync if needed) before copy_tensor","Ensure the preprocessing stage is configured for GPU (e.g. move image encode onto the target device)","Add an assert tensor.is_cuda in debug builds near the producer"],"exampleFix":"# before\nlease, view = pool.copy_tensor(cpu_tensor)\n# after\nlease, view = pool.copy_tensor(cpu_tensor.to('cuda', non_blocking=True))\ntorch.cuda.synchronize()","handlingStrategy":"type-guard","validationCode":"if not tensor.is_cuda:\n    tensor = tensor.to(device, non_blocking=True)\n    torch.cuda.synchronize()","typeGuard":"def ensure_cuda(t: torch.Tensor, device: str) -> torch.Tensor:\n    return t if t.is_cuda else t.to(device)","tryCatchPattern":null,"preventionTips":["Move multimodal preprocessing outputs to the target GPU device at the stage boundary","Assert tensor.is_cuda in tests around transport calls"],"tags":["multimodal","cuda","tensor-device"],"backgroundTag":"tensor-device-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}