vllm-project/vllm · error · ValueError

Unsupported dtype {dtype}: should be one of int8, uint8, int

Error message

Unsupported dtype {dtype}: should be one of int8, uint8, int32, int64, float16, float32, float64, bfloat16, float8e4m3.

What it means

ncclDataType.from_torch maps a torch.dtype to an NCCL datatype enum via a fixed table. Only int8, uint8, int32, int64, float16, float32, float64, bfloat16 and float8e4m3 are mapped; any other dtype has no NCCL equivalent and raises ValueError listing the supported set.

Source

Thrown at vllm/distributed/device_communicators/pynccl_wrapper.py:127

            torch.float64: cls.ncclFloat64,
            torch.bfloat16: cls.ncclBfloat16,
            current_platform.fp8_dtype(): cls.ncclFloat8e4m3,
        }

    @classmethod
    def supports_torch_dtype(cls, dtype: torch.dtype) -> bool:
        return dtype in cls._torch_to_nccl_map()

    @classmethod
    def try_from_torch(cls, dtype: torch.dtype) -> int | None:
        return cls._torch_to_nccl_map().get(dtype)

    @classmethod
    def from_torch(cls, dtype: torch.dtype) -> int:
        nccl_dtype = cls.try_from_torch(dtype)
        if nccl_dtype is not None:
            return nccl_dtype
        raise ValueError(
            f"Unsupported dtype {dtype}: should be one of "
            f"int8, uint8, int32, int64, float16, float32, float64, bfloat16,"
            " float8e4m3."
        )


ncclRedOp_t = ctypes.c_int


class ncclRedOpTypeEnum:
    ncclSum = 0
    ncclProd = 1
    ncclMax = 2
    ncclMin = 3
    ncclAvg = 4
    ncclNumOps = 5

    @classmethod

View on GitHub (pinned to c794754062)

Solutions

1. Cast the tensor to a supported dtype before the collective (e.g. t.to(torch.int32) or t.to(torch.bfloat16)) and cast back after
2. Use ncclDataType.try_from_torch(dtype) first to detect the gap and pick a fallback dtype programmatically
3. For fp8, use float8_e4m3fn (mapped) rather than e5m2 variants

Example fix

# before
pynccl.all_reduce(x)  # x.dtype == torch.int16 -> ValueError

# after
x = x.to(torch.int32)
pynccl.all_reduce(x)
x = x.to(torch.int16)

from vllm.distributed.device_communicators.pynccl_wrapper import ncclDataType
SUPPORTED = {torch.int8, torch.uint8, torch.int32, torch.int64,
             torch.float16, torch.float32, torch.float64, torch.bfloat16, torch.float8_e4m3fn}
assert tensor.dtype in SUPPORTED, f"cast {tensor.dtype} before pynccl collective"

def nccl_compatible(dtype: torch.dtype) -> bool:
    return ncclDataType.try_from_torch(dtype) is not None

try:
    ncclDataType.from_torch(x.dtype)
except ValueError:
    x = x.to(torch.float16)  # or another supported dtype

When it happens

Trigger: Passing a tensor whose dtype is torch.int16, torch.bool, torch.complex64/128, torch.float8_e5m2, or a torch._scaled_gamma style dtype into a PyNccl collective (all_reduce, reduce_scatter, all_gather through vllm.distributed device communicators) which calls ncclDataType.from_torch(tensor.dtype).

Common situations: Optimizer-state or KV-cache tensors carried as int16/bool; experimental fp8 variants (e5m2) not yet registered; user collectives on custom casted dtypes; a model passing uint4/int4 packed weights through a collective.

Related errors

• Unsupported op: {op}
• Expected 128 bytes for ncclUniqueId, got {len(data)} bytes
• Unknown dtype: {dtype!r}
• Unknown dtype: {dtype}
• Unknown dtype: {head_dtype!r}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/da61a4595a2ef068. Report an issue: GitHub.