Lightning-AI/pytorch-lightning · critical · RuntimeError

Torch distributed is not available.

Error message

Torch distributed is not available.

What it means

`TorchCollective.__init__` raises RuntimeError when `torch.distributed.is_available()` is False, i.e. the installed PyTorch build was compiled without distributed support (no NCCL/Gloo support on that platform/build).

Source

Thrown at src/lightning/fabric/plugins/collectives/torch_collective.py:32

    from torch.distributed.constants import default_pg_timeout
else:
    default_pg_timeout = datetime.timedelta(seconds=1800)


class TorchCollective(Collective):
    """Collective operations using `torch.distributed <https://pytorch.org/docs/stable/distributed.html>`__.

    .. warning:: This is an :ref:`experimental <versioning:Experimental API>` feature which is still in development.

    """

    manages_default_group = False
    addr_key = "MASTER_ADDR"
    port_key = "MASTER_PORT"

    def __init__(self) -> None:
        if not dist.is_available():
            raise RuntimeError("Torch distributed is not available.")
        super().__init__()

    @property
    @override
    def group(self) -> CollectibleGroup:
        if self._group is None:
            self._group = dist.GroupMember.WORLD
        return super().group

    @property
    @override
    def rank(self) -> int:
        # local rank
        return dist.get_rank(self.group)  # type: ignore[arg-type]

    @property
    @override
    def world_size(self) -> int:

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Install a PyTorch build with distributed support, typically the official Linux wheels: `pip install torch` from pytorch.org
2. Use a container/Linux environment for distributed training
3. If distributed is optional, fall back to non-distributed Fabric strategy when `torch.distributed.is_available()` is False

Example fix

# before
collective = TorchCollective()  # RuntimeError on non-distributed build

# after
import torch.distributed as dist
if dist.is_available():
    collective = TorchCollective()
else:
    strategy = SingleDeviceStrategy(...)  # run without distributed

import torch.distributed as dist
assert dist.is_available(), 'install a torch build with distributed support'

When it happens

Trigger: Constructing `TorchCollective()` under a PyTorch build lacking distributed (common on some Windows builds, very old/edge builds, or stripped-down wheels).

Common situations: Developing on Windows where torch.distributed has limited availability; CI using minimal torch wheels; running Fabric's distributed features on unsupported platforms.

Related errors

• `{type(self).__name__}` does not own a group. HINT: try `col
• `{type(self).__name__}` already owns a group.
• `{type(self).__name__}` does not own a group to destroy.
• Unsupported op {op!r} of type {type(op).__name__}
• op {op!r} is not a member of `ReduceOp`

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/1280b3bd24b03fe6. Report an issue: GitHub.