Lightning-AI/pytorch-lightning · error · ValueError

Unsupported op {op!r} of type {type(op).__name__}

Error message

Unsupported op {op!r} of type {type(op).__name__}

What it means

`_convert_to_native_op` converts the `op` argument of all_reduce/reduce/reduce_scatter into a native `ReduceOp`/`RedOpType`. If `op` is neither a ReduceOp/RedOpType instance nor a string, it raises this ValueError — only those two types are accepted.

Source

Thrown at src/lightning/fabric/plugins/collectives/torch_collective.py:213

    @classmethod
    @override
    def destroy_group(cls, group: CollectibleGroup) -> None:
        # can be called by all processes in the default group, group will be `object()` if they are not part of the
        # current group
        if group in dist.distributed_c10d._pg_map:
            dist.destroy_process_group(group)

    @classmethod
    @override
    def _convert_to_native_op(cls, op: Union[str, ReduceOp, RedOpType]) -> Union[ReduceOp, RedOpType]:
        # `ReduceOp` is an empty shell for `RedOpType`, the latter being the actually returned class.
        # For example, `ReduceOp.SUM` returns a `RedOpType.SUM`. the only exception is `RedOpType.PREMUL_SUM` where
        # `ReduceOp` is still the desired class, but it's created via a special `_make_nccl_premul_sum` function
        if isinstance(op, (ReduceOp, RedOpType)):
            return op
        if not isinstance(op, str):
            raise ValueError(f"Unsupported op {op!r} of type {type(op).__name__}")
        op = op.upper()
        # `ReduceOp` should contain `RedOpType`'s members
        value = getattr(ReduceOp, op, None)
        if value is None:
            raise ValueError(f"op {op!r} is not a member of `ReduceOp`")
        return value

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Pass a `torch.distributed.ReduceOp` member directly, e.g. `op=torch.distributed.ReduceOp.SUM`
2. Or pass its name as a string: `op='sum'` / `op='MAX'` (it is uppercased and looked up on ReduceOp)
3. Remove custom int/op-code values; there is no mapping from raw ints in this API

Example fix

# before
collective.all_reduce(tensor, op=2)  # int not supported

# after
import torch.distributed as dist
collective.all_reduce(tensor, op=dist.ReduceOp.SUM)
# or: collective.all_reduce(tensor, op='sum')

import torch.distributed as dist
assert isinstance(op, (str, dist.ReduceOp)), 'op must be str or ReduceOp'

import torch.distributed as dist
def is_valid_op(op) -> bool:
    return isinstance(op, str) or isinstance(op, (dist.ReduceOp, dist.ReduceOp.RedOpType))

When it happens

Trigger: Passing an int raw op code, None, a custom enum, or an object as `op=` to `collective.all_reduce(tensor, op=...)`, `reduce`, or `reduce_scatter`.

Common situations: Porting code that used raw integer op codes from older torch.distributed APIs; passing `dist.ReduceOp.SUM` from a mismatched torch namespace or a different library's enum; typos causing unexpected object types.

Related errors

• op {op!r} is not a member of `ReduceOp`
• Only PyTorch DataLoader are currently supported in `setup_da
• you tried to log {v} which is currently not supported. Try a
• `{type(self).__name__}` does not own a group. HINT: try `col
• `{type(self).__name__}` already owns a group.

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/c895d11b077eb6d9. Report an issue: GitHub.