{"record":{"id":"da61a4595a2ef068","repo":"vllm-project/vllm","slug":"unsupported-dtype-dtype-should-be-one-of-int8","errorCode":null,"errorMessage":"Unsupported dtype {dtype}: should be one of int8, uint8, int32, int64, float16, float32, float64, bfloat16, float8e4m3.","messagePattern":"Unsupported dtype (.+?): should be one of int8, uint8, int32, int64, float16, float32, float64, bfloat16, float8e4m3\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/device_communicators/pynccl_wrapper.py","lineNumber":127,"sourceCode":"            torch.float64: cls.ncclFloat64,\n            torch.bfloat16: cls.ncclBfloat16,\n            current_platform.fp8_dtype(): cls.ncclFloat8e4m3,\n        }\n\n    @classmethod\n    def supports_torch_dtype(cls, dtype: torch.dtype) -> bool:\n        return dtype in cls._torch_to_nccl_map()\n\n    @classmethod\n    def try_from_torch(cls, dtype: torch.dtype) -> int | None:\n        return cls._torch_to_nccl_map().get(dtype)\n\n    @classmethod\n    def from_torch(cls, dtype: torch.dtype) -> int:\n        nccl_dtype = cls.try_from_torch(dtype)\n        if nccl_dtype is not None:\n            return nccl_dtype\n        raise ValueError(\n            f\"Unsupported dtype {dtype}: should be one of \"\n            f\"int8, uint8, int32, int64, float16, float32, float64, bfloat16,\"\n            \" float8e4m3.\"\n        )\n\n\nncclRedOp_t = ctypes.c_int\n\n\nclass ncclRedOpTypeEnum:\n    ncclSum = 0\n    ncclProd = 1\n    ncclMax = 2\n    ncclMin = 3\n    ncclAvg = 4\n    ncclNumOps = 5\n\n    @classmethod","sourceCodeStart":109,"sourceCodeEnd":145,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/device_communicators/pynccl_wrapper.py#L109-L145","documentation":"ncclDataType.from_torch maps a torch.dtype to an NCCL datatype enum via a fixed table. Only int8, uint8, int32, int64, float16, float32, float64, bfloat16 and float8e4m3 are mapped; any other dtype has no NCCL equivalent and raises ValueError listing the supported set.","triggerScenarios":"Passing a tensor whose dtype is torch.int16, torch.bool, torch.complex64/128, torch.float8_e5m2, or a torch._scaled_gamma style dtype into a PyNccl collective (all_reduce, reduce_scatter, all_gather through vllm.distributed device communicators) which calls ncclDataType.from_torch(tensor.dtype).","commonSituations":"Optimizer-state or KV-cache tensors carried as int16/bool; experimental fp8 variants (e5m2) not yet registered; user collectives on custom casted dtypes; a model passing uint4/int4 packed weights through a collective.","solutions":["Cast the tensor to a supported dtype before the collective (e.g. t.to(torch.int32) or t.to(torch.bfloat16)) and cast back after","Use ncclDataType.try_from_torch(dtype) first to detect the gap and pick a fallback dtype programmatically","For fp8, use float8_e4m3fn (mapped) rather than e5m2 variants"],"exampleFix":"# before\npynccl.all_reduce(x)  # x.dtype == torch.int16 -> ValueError\n\n# after\nx = x.to(torch.int32)\npynccl.all_reduce(x)\nx = x.to(torch.int16)","handlingStrategy":"type-guard","validationCode":"from vllm.distributed.device_communicators.pynccl_wrapper import ncclDataType\nSUPPORTED = {torch.int8, torch.uint8, torch.int32, torch.int64,\n             torch.float16, torch.float32, torch.float64, torch.bfloat16, torch.float8_e4m3fn}\nassert tensor.dtype in SUPPORTED, f\"cast {tensor.dtype} before pynccl collective\"","typeGuard":"def nccl_compatible(dtype: torch.dtype) -> bool:\n    return ncclDataType.try_from_torch(dtype) is not None","tryCatchPattern":"try:\n    ncclDataType.from_torch(x.dtype)\nexcept ValueError:\n    x = x.to(torch.float16)  # or another supported dtype","preventionTips":["Feature-detect with ncclDataType.try_from_torch before collectives","Standardize internal tensors on mapped dtypes","Add dtype asserts in unit tests for custom collectives"],"tags":["nccl","dtype","distributed","validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}