vllm-project/vllm · error · ValueError

Expected 128 bytes for ncclUniqueId, got {len(data)} bytes

Error message

Expected 128 bytes for ncclUniqueId, got {len(data)} bytes

What it means

NCCLLibrary.unique_id_from_bytes reconstructs an ncclUniqueId from a serialized blob. NCCL unique ids are a fixed 128-byte structure (NCCL_UNIQUE_ID_BYTES), so the loader validates len(data) == 128 before ctypes.memmove; any other length is rejected rather than read out of bounds.

Source

Thrown at vllm/distributed/device_communicators/pynccl_wrapper.py:440

        # something like 21903
        return version.value

    def ncclGetVersion(self) -> str:
        version_str = str(self.ncclGetRawVersion())
        # something like 21903 --> "2.19.3"
        major = version_str[0].lstrip("0")
        minor = version_str[1:3].lstrip("0")
        patch = version_str[3:].lstrip("0")
        return f"{major}.{minor}.{patch}"

    def ncclGetUniqueId(self) -> ncclUniqueId:
        unique_id = ncclUniqueId()
        self.NCCL_CHECK(self._funcs["ncclGetUniqueId"](ctypes.byref(unique_id)))
        return unique_id

    def unique_id_from_bytes(self, data: bytes) -> ncclUniqueId:
        if len(data) != 128:
            raise ValueError(
                f"Expected 128 bytes for ncclUniqueId, got {len(data)} bytes"
            )
        unique_id = ncclUniqueId()
        ctypes.memmove(ctypes.addressof(unique_id.internal), data, 128)
        return unique_id

    def ncclCommInitRank(
        self, world_size: int, unique_id: ncclUniqueId, rank: int
    ) -> ncclComm_t:
        comm = ncclComm_t()
        self.NCCL_CHECK(
            self._funcs["ncclCommInitRank"](
                ctypes.byref(comm), world_size, unique_id, rank
            )
        )
        return comm

    def ncclAllReduce(

View on GitHub (pinned to c794754062)

Solutions

1. Send exactly bytes(unique_id.internal) (128 raw bytes) and feed that back into unique_id_from_bytes
2. If transporting as text, hex-encode and unhexlify on the other side: bytes.fromhex(uid_hex)
3. Length-check the payload before calling: assert len(data) == 128

Example fix

# before
uid_hex = bytes(uid.internal).hex()
lib.unique_id_from_bytes(uid_hex)  # len 256 -> ValueError

# after
from binascii import unhexlify
lib.unique_id_from_bytes(unhexlify(uid_hex))  # 128 bytes

assert isinstance(data, (bytes, bytearray)) and len(data) == 128, (
    f"ncclUniqueId payload must be exactly 128 raw bytes, got {type(data).__name__} len={len(data) if isinstance(data, (bytes, bytearray)) else 'n/a'}")

def is_valid_nccl_unique_id_bytes(data: object) -> bool:
    return isinstance(data, (bytes, bytearray)) and len(data) == 128

When it happens

Trigger: Passing the hex-encoded string of the id (256 chars) instead of raw bytes; passing a Python list/array of ints; slicing a combined broadcast payload incorrectly so the id segment is truncated or includes extra bytes; a string that was never .encode()d/decoded consistently across ranks.

Common situations: Hand-rolling unique-id exchange over Ray actors, HTTP, or a store where bytes get re-encoded (str(uid) on Python 2/3 mixing, base64 without decode); versions of a launcher that changed the serialization format.

Related errors

• Unsupported dtype {dtype}: should be one of int8, uint8, int
• Unsupported op: {op}
• {self.communicator} communicator is incompatible with async
• DeepEPv2 communicator properties query failed; networking ca
• DeepEPv2 requires NCCL GIN (GPU-Initiated Networking). This

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/ee050b9eeff42440. Report an issue: GitHub.