What causes "Unsupported op: {op}" in vllm-project/vllm?

Use one of ReduceOp.SUM, PRODUCT, MAX, MIN or AVG for collectives routed through PyNccl

vllm-project/vllm · error · ValueError

Unsupported op: {op}

Error message

Unsupported op: {op}

What it means

ncclRedOpTypeEnum.from_torch translates a torch.distributed.ReduceOp into an NCCL reduction operator. Only SUM, PRODUCT, MAX, MIN and AVG are mapped; NCCL has no enum for bitwise or other torch ops, so anything else raises ValueError('Unsupported op: {op}').

Source

Thrown at vllm/distributed/device_communicators/pynccl_wrapper.py:157

    ncclProd = 1
    ncclMax = 2
    ncclMin = 3
    ncclAvg = 4
    ncclNumOps = 5

    @classmethod
    def from_torch(cls, op: ReduceOp) -> int:
        if op == ReduceOp.SUM:
            return cls.ncclSum
        if op == ReduceOp.PRODUCT:
            return cls.ncclProd
        if op == ReduceOp.MAX:
            return cls.ncclMax
        if op == ReduceOp.MIN:
            return cls.ncclMin
        if op == ReduceOp.AVG:
            return cls.ncclAvg
        raise ValueError(f"Unsupported op: {op}")


@dataclass
class Function:
    name: str
    restype: Any
    argtypes: list[Any]


class NCCLLibrary:
    exported_functions = [
        # const char* ncclGetErrorString(ncclResult_t result)
        Function("ncclGetErrorString", ctypes.c_char_p, [ncclResult_t]),
        # ncclResult_t  ncclGetVersion(int *version);
        Function("ncclGetVersion", ncclResult_t, [ctypes.POINTER(ctypes.c_int)]),
        # ncclResult_t ncclGetUniqueId(ncclUniqueId* uniqueId);
        Function("ncclGetUniqueId", ncclResult_t, [ctypes.POINTER(ncclUniqueId)]),
        # ncclResult_t  ncclCommInitRank(

View on GitHub (pinned to c794754062)

Solutions

Use one of ReduceOp.SUM, PRODUCT, MAX, MIN or AVG for collectives routed through PyNccl
For bitwise data, pack bits into int32 and use MAX/SUM semantics that reproduce the intended result
Route bitwise collectives through torch.distributed (gloo/nccl process group) instead of the pynccl wrapper

Example fix

# before
pynccl.all_reduce(x, op=ReduceOp.BOR)  # ValueError

# after
# bitwise-or of 0/1 flags == max
pynccl.all_reduce(x, op=ReduceOp.MAX)

Defensive patterns

Strategy: type-guard

Validate before calling

SUPPORTED_OPS = {ReduceOp.SUM, ReduceOp.PRODUCT, ReduceOp.MAX, ReduceOp.MIN, ReduceOp.AVG}
assert op in SUPPORTED_OPS, f"{op} has no NCCL equivalent; pick SUM/PRODUCT/MAX/MIN/AVG"

Type guard

def nccl_op_supported(op: ReduceOp) -> bool:
    return op in {ReduceOp.SUM, ReduceOp.PRODUCT, ReduceOp.MAX, ReduceOp.MIN, ReduceOp.AVG}

Try / catch

try:
    ncclRedOpTypeEnum.from_torch(op)
except ValueError:
    op = ReduceOp.SUM  # choose a semantically valid fallback

Prevention

Avoid bitwise ReduceOps on pynccl paths
Wrap op selection behind a whitelist helper
Unit-test collective helpers against all ReduceOp members

When it happens

Trigger: Calling a PyNccl-backed collective with op=ReduceOp.BAND / BOR / BXOR (no NCCL equivalent), or passing the ReduceOp handle itself instead of a member (op=ReduceOp instead of ReduceOp.SUM).

Common situations: Porting torch.distributed code that used bitwise reduce ops for mask/flag tensors; passing a default ReduceOp object from a generic wrapper that assumed a different enum surface.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/600f2f81a9e30c16. Report an issue: GitHub.