{"record":{"id":"1f3a7de177a9f330","repo":"sgl-project/sglang","slug":"byte-tensor-must-be-a-sufficiently-large-contiguou","errorCode":null,"errorMessage":"byte_tensor must be a sufficiently large contiguous uint8 tensor on cuda:{device_id}","messagePattern":"byte_tensor must be a sufficiently large contiguous uint8 tensor on cuda:(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/multimodal/transport/memory_pool.py","lineNumber":186,"sourceCode":"        transport_name: str,\n        max_inflight_slices: int = DEFAULT_MAX_INFLIGHT_SLICES,\n    ) -> None:\n        if memory_size <= 0:\n            raise ValueError(\"memory_size must be positive\")\n        if consumer_count <= 0:\n            raise ValueError(\"consumer_count must be positive\")\n        if max_inflight_slices <= 0:\n            raise ValueError(\"max_inflight_slices must be positive\")\n        if recycle_interval <= 0:\n            raise ValueError(\"recycle_interval must be positive\")\n        if (\n            not byte_tensor.is_cuda\n            or byte_tensor.device.index != device_id\n            or byte_tensor.dtype != torch.uint8\n            or not byte_tensor.is_contiguous()\n            or byte_tensor.numel() < memory_size\n        ):\n            raise ValueError(\n                \"byte_tensor must be a sufficiently large contiguous uint8 tensor \"\n                f\"on cuda:{device_id}\"\n            )\n\n        self.memory_size = memory_size\n        self.byte_tensor = byte_tensor\n        self.base_address = base_address\n        self.device_id = device_id\n        self.consumer_count = consumer_count\n        self.control_words_per_slot = 1 + consumer_count\n        self.max_inflight_slices = max_inflight_slices\n        self.transport_name = transport_name\n        control_bytes = (\n            max_inflight_slices * self.control_words_per_slot * CONTROL_WORD_BYTES\n        )\n        self.data_start = align_up(control_bytes, DATA_ALIGNMENT)\n        if memory_size <= self.data_start:\n            raise ValueError(","sourceCodeStart":168,"sourceCodeEnd":204,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/multimodal/transport/memory_pool.py#L168-L204","documentation":"The pool must be backed by a CUDA uint8 byte tensor that lives on the exact device (device_id), is contiguous, and has at least memory_size elements. Any mismatch (CPU tensor, wrong device index, wrong dtype, non-contiguous, or too small) fails this combined check because the pool does raw pointer arithmetic over that buffer.","triggerScenarios":"Passing a torch.zeros(n, dtype=torch.uint8, device='cpu'); a tensor on cuda:1 when device_id=0; a float16 view; a strided/permuted tensor; or numel < memory_size.","commonSituations":"Tensor allocated before torch.cuda.set_device, device index taken from a different rank in TP, or reusing a pooled buffer from another process/device.","solutions":["Allocate with torch.empty(memory_size, dtype=torch.uint8, device=f'cuda:{device_id}')","Verify byte_tensor.device.index == device_id and dtype is torch.uint8","Ensure the tensor is contiguous and numel >= memory_size"],"exampleFix":"# before\nbuf = torch.zeros(memory_size, dtype=torch.uint8)  # CPU\n# after\nbuf = torch.zeros(memory_size, dtype=torch.uint8, device=f'cuda:{device_id}')","handlingStrategy":"type-guard","validationCode":"assert byte_tensor.is_cuda and byte_tensor.device.index == device_id\nassert byte_tensor.dtype == torch.uint8 and byte_tensor.is_contiguous()\nassert byte_tensor.numel() >= memory_size","typeGuard":"def is_valid_pool_buffer(t: torch.Tensor, device_id: int, size: int) -> bool:\n    return (t.is_cuda and t.device.index == device_id\n            and t.dtype == torch.uint8 and t.is_contiguous()\n            and t.numel() >= size)","tryCatchPattern":null,"preventionTips":["Allocate the backing buffer with an explicit device=f'cuda:{device_id}' argument","Never reuse buffers across ranks/devices without re-checking device.index"],"tags":["multimodal","cuda","tensor-validation","memory-pool"],"backgroundTag":"tensor-device-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}