{"record":{"id":"0d95585d59cfa110","repo":"Lightning-AI/pytorch-lightning","slug":"torch-distributed-is-not-available-cannot-initial","errorCode":null,"errorMessage":"torch.distributed is not available. Cannot initialize distributed process group","messagePattern":"torch\\.distributed is not available\\. Cannot initialize distributed process group","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"src/lightning/fabric/utilities/distributed.py","lineNumber":273,"sourceCode":"    **kwargs: Any,\n) -> None:\n    \"\"\"Utility function to initialize distributed connection by setting env variables and initializing the distributed\n    process group.\n\n    Args:\n        cluster_environment: ``ClusterEnvironment`` instance\n        torch_distributed_backend: Backend to use (includes `nccl` and `gloo`)\n        global_rank: Rank of the current process\n        world_size: Number of processes in the group\n        kwargs: Kwargs for ``init_process_group``\n\n    Raises:\n        RuntimeError:\n            If ``torch.distributed`` is not available\n\n    \"\"\"\n    if not torch.distributed.is_available():\n        raise RuntimeError(\"torch.distributed is not available. Cannot initialize distributed process group\")\n    if torch.distributed.is_initialized():\n        log.debug(\"torch.distributed is already initialized. Exiting early\")\n        return\n    global_rank = global_rank if global_rank is not None else cluster_environment.global_rank()\n    world_size = world_size if world_size is not None else cluster_environment.world_size()\n    os.environ[\"MASTER_ADDR\"] = cluster_environment.main_address\n    os.environ[\"MASTER_PORT\"] = str(cluster_environment.main_port)\n    log.info(f\"Initializing distributed: GLOBAL_RANK: {global_rank}, MEMBER: {global_rank + 1}/{world_size}\")\n    torch.distributed.init_process_group(torch_distributed_backend, rank=global_rank, world_size=world_size, **kwargs)\n\n    if torch_distributed_backend == \"nccl\":\n        # PyTorch >= 2.4 warns about undestroyed NCCL process group, so we need to do it at program exit\n        atexit.register(_destroy_dist_connection)\n\n    # On rank=0 let everyone know training is starting\n    rank_zero_info(\n        f\"{'-' * 100}\\n\"\n        f\"distributed_backend={torch_distributed_backend}\\n\"","sourceCodeStart":255,"sourceCodeEnd":291,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/utilities/distributed.py#L255-L291","documentation":"Lightning's distributed setup calls _init_dist_connection to create the torch.distributed process group; if torch.distributed.is_available() is False (PyTorch built without distributed support, e.g. Windows builds or stripped wheels), it raises RuntimeError before any rendezvous happens. It is a build/dependency problem, not a topology problem.","triggerScenarios":"Running multi-device Fabric/Trainer (devices>1 or a distributed strategy) on a PyTorch build lacking torch.distributed; some pip Windows wheels and old conda builds; using parallel strategy in an environment where 'python -c \"import torch.distributed\"' fails.","commonSituations":"Windows laptops trying DDP without WSL; minimal/embedded PyTorch wheels; version mismatches after upgrading torch; CI images with CPU-only builds.","solutions":["Verify availability: python -c \"import torch; print(torch.distributed.is_available())\"","Install a PyTorch build with distributed support (e.g. the default Linux CUDA/CPU wheels, or use WSL2 on Windows)","If distributed isn't needed, use a single device or strategy='ddp' replaced by non-distributed setup (devices=1 / no parallel strategy)"],"exampleFix":"# before\nfabric = Fabric(strategy=\"ddp\", devices=2)  # torch.distributed unavailable\n\n# after (WSL/Linux with proper torch), or:\nfabric = Fabric(devices=1)  # no process group needed","handlingStrategy":"validation","validationCode":"import torch\n\nif not torch.distributed.is_available():\n    raise SystemExit(\"This PyTorch build lacks torch.distributed; reinstall a full wheel or run single-device\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Smoke-test torch.distributed.is_available() in CI before distributed runs","Use standard PyTorch wheels (Linux/WSL2) for DDP workloads"],"tags":["distributed","pytorch","environment","runtimeerror"],"backgroundTag":"distributed-backend-unavailable","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}