What causes "Unsupported op: {op}" in vllm-project/vllm?

Use one of ReduceOp.SUM, PRODUCT, MAX, MIN or AVG for collectives routed through PyNccl

vllm-project/vllm · error · ValueError

Unsupported op

Error message

Unsupported op: {op}

What it means

ncclRedOpTypeEnum.from_torch translates a torch.distributed.ReduceOp into an NCCL reduction operator. Only SUM, PRODUCT, MAX, MIN and AVG are mapped; NCCL has no enum for bitwise or other torch ops, so anything else raises ValueError('Unsupported op: {op}').

Solutions

Use one of ReduceOp.SUM, PRODUCT, MAX, MIN or AVG for collectives routed through PyNccl
For bitwise data, pack bits into int32 and use MAX/SUM semantics that reproduce the intended result
Route bitwise collectives through torch.distributed (gloo/nccl process group) instead of the pynccl wrapper

Example fix

# before
pynccl.all_reduce(x, op=ReduceOp.BOR)  # ValueError

# after
# bitwise-or of 0/1 flags == max
pynccl.all_reduce(x, op=ReduceOp.MAX)

Defensive patterns

Strategy: type-guard

Validate before calling

SUPPORTED_OPS = {ReduceOp.SUM, ReduceOp.PRODUCT, ReduceOp.MAX, ReduceOp.MIN, ReduceOp.AVG}
assert op in SUPPORTED_OPS, f"{op} has no NCCL equivalent; pick SUM/PRODUCT/MAX/MIN/AVG"

Type guard

def nccl_op_supported(op: ReduceOp) -> bool:
    return op in {ReduceOp.SUM, ReduceOp.PRODUCT, ReduceOp.MAX, ReduceOp.MIN, ReduceOp.AVG}

Try / catch

try:
    ncclRedOpTypeEnum.from_torch(op)
except ValueError:
    op = ReduceOp.SUM  # choose a semantically valid fallback

Prevention

Avoid bitwise ReduceOps on pynccl paths
Wrap op selection behind a whitelist helper
Unit-test collective helpers against all ReduceOp members

When it happens

Trigger: Calling a PyNccl-backed collective with op=ReduceOp.BAND / BOR / BXOR (no NCCL equivalent), or passing the ReduceOp handle itself instead of a member (op=ReduceOp instead of ReduceOp.SUM).

Common situations: Porting torch.distributed code that used bitwise reduce ops for mask/flag tensors; passing a default ReduceOp object from a generic wrapper that assumed a different enum surface.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/600f2f81a9e30c16. Report an issue: GitHub.

Appendix: source

Thrown at vllm/distributed/device_communicators/pynccl_wrapper.py:157

    ncclProd = 1
    ncclMax = 2
    ncclMin = 3
    ncclAvg = 4
    ncclNumOps = 5

    @classmethod
    def from_torch(cls, op: ReduceOp) -> int:
        if op == ReduceOp.SUM:
            return cls.ncclSum
        if op == ReduceOp.PRODUCT:
            return cls.ncclProd
        if op == ReduceOp.MAX:
            return cls.ncclMax
        if op == ReduceOp.MIN:
            return cls.ncclMin
        if op == ReduceOp.AVG:
            return cls.ncclAvg
        raise ValueError(f"Unsupported op: {op}")


@dataclass
class Function:
    name: str
    restype: Any
    argtypes: list[Any]


class NCCLLibrary:
    exported_functions = [
        # const char* ncclGetErrorString(ncclResult_t result)
        Function("ncclGetErrorString", ctypes.c_char_p, [ncclResult_t]),
        # ncclResult_t  ncclGetVersion(int *version);
        Function("ncclGetVersion", ncclResult_t, [ctypes.POINTER(ctypes.c_int)]),
        # ncclResult_t ncclGetUniqueId(ncclUniqueId* uniqueId);
        Function("ncclGetUniqueId", ncclResult_t, [ctypes.POINTER(ncclUniqueId)]),
        # ncclResult_t  ncclCommInitRank(

View on GitHub (pinned to c794754062)