Lightning-AI/pytorch-lightning · error · ValueError

op {op!r} is not a member of `ReduceOp`

Error message

op {op!r} is not a member of `ReduceOp`

What it means

After uppercasing a string `op`, `_convert_to_native_op` looks it up via `getattr(ReduceOp, op, None)`; if the name is not a member of `torch.distributed.ReduceOp` (e.g. 'avg' where unsupported, or a typo), this ValueError is raised.

Source

Thrown at src/lightning/fabric/plugins/collectives/torch_collective.py:218

        # current group
        if group in dist.distributed_c10d._pg_map:
            dist.destroy_process_group(group)

    @classmethod
    @override
    def _convert_to_native_op(cls, op: Union[str, ReduceOp, RedOpType]) -> Union[ReduceOp, RedOpType]:
        # `ReduceOp` is an empty shell for `RedOpType`, the latter being the actually returned class.
        # For example, `ReduceOp.SUM` returns a `RedOpType.SUM`. the only exception is `RedOpType.PREMUL_SUM` where
        # `ReduceOp` is still the desired class, but it's created via a special `_make_nccl_premul_sum` function
        if isinstance(op, (ReduceOp, RedOpType)):
            return op
        if not isinstance(op, str):
            raise ValueError(f"Unsupported op {op!r} of type {type(op).__name__}")
        op = op.upper()
        # `ReduceOp` should contain `RedOpType`'s members
        value = getattr(ReduceOp, op, None)
        if value is None:
            raise ValueError(f"op {op!r} is not a member of `ReduceOp`")
        return value

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Use the exact ReduceOp member name, e.g. `op='sum'`, `op='max'`, `op='min'`, `op='product'` — or better, pass `torch.distributed.ReduceOp.SUM` directly
2. Print/inspect valid members: `dir(torch.distributed.ReduceOp)`
3. Update the string for your torch version (member sets changed across releases)

Example fix

# before
collective.all_reduce(tensor, op='meansum')  # not a member

# after
collective.all_reduce(tensor, op='sum')
# or
import torch.distributed as dist
collective.all_reduce(tensor, op=dist.ReduceOp.SUM)

import torch.distributed as dist
assert str(op).upper() in dir(dist.ReduceOp), f'{op!r} not a ReduceOp member'

import torch.distributed as dist
def is_known_op(name: str) -> bool:
    return getattr(dist.ReduceOp, name.upper(), None) is not None

When it happens

Trigger: Passing a string like `op='product'`, `op='avg'` (not a member on some backends/torch versions), or a misspelling such as `op='suM'` that resolves to a nonexistent member, to all_reduce/reduce/reduce_scatter.

Common situations: Assuming all backends expose the same reduce-op names (NCCL vs Gloo differences across torch versions); copying op strings from other frameworks.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Unsupported op {op!r} of type {type(op).__name__}
• `{type(self).__name__}` does not own a group. HINT: try `col
• `{type(self).__name__}` already owns a group.
• `{type(self).__name__}` does not own a group to destroy.
• Torch distributed is not available.

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/493157dcaed7ad84. Report an issue: GitHub.