{"record":{"id":"d1f8ac54b55608c8","repo":"Lightning-AI/pytorch-lightning","slug":"accessing-the-device-mesh-before-processes-have-in","errorCode":null,"errorMessage":"Accessing the device mesh before processes have initialized is not allowed.","messagePattern":"Accessing the device mesh before processes have initialized is not allowed\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/lightning/fabric/strategies/model_parallel.py","lineNumber":120,"sourceCode":"        process_group_backend: Optional[str] = None,\n        timeout: Optional[timedelta] = default_pg_timeout,\n    ) -> None:\n        super().__init__()\n        self._parallelize_fn = parallelize_fn\n        self._data_parallel_size = data_parallel_size\n        self._tensor_parallel_size = tensor_parallel_size\n        self._num_nodes = 1\n        self._save_distributed_checkpoint = save_distributed_checkpoint\n        self._process_group_backend: Optional[str] = process_group_backend\n        self._timeout: Optional[timedelta] = timeout\n        self._backward_sync_control = _ParallelBackwardSyncControl()\n\n        self._device_mesh: Optional[DeviceMesh] = None\n\n    @property\n    def device_mesh(self) -> \"DeviceMesh\":\n        if self._device_mesh is None:\n            raise RuntimeError(\"Accessing the device mesh before processes have initialized is not allowed.\")\n        return self._device_mesh\n\n    @property\n    @override\n    def checkpoint_io(self) -> CheckpointIO:\n        raise NotImplementedError(f\"The `{type(self).__name__}` does not use the `CheckpointIO` plugin interface.\")\n\n    @checkpoint_io.setter\n    @override\n    def checkpoint_io(self, io: CheckpointIO) -> None:\n        raise NotImplementedError(f\"The `{type(self).__name__}` does not support setting a `CheckpointIO` plugin.\")\n\n    @property\n    @override\n    def root_device(self) -> torch.device:\n        assert self.parallel_devices is not None\n        return self.parallel_devices[self.local_rank]\n","sourceCodeStart":102,"sourceCodeEnd":138,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/strategies/model_parallel.py#L102-L138","documentation":"ModelParallelStrategy lazily creates its DeviceMesh during environment setup (setup_environment). The device_mesh property raises if accessed before that setup ran, because no process group or mesh exists yet. Any code touching strategy.device_mesh (or attributes deriving from it) prior to Fabric's setup phase triggers this error.","triggerScenarios":"Accessing strategy.device_mesh in __init__, at config time, or before calling fabric.setup()/run() (i.e., before setup_environment executes) on ModelParallelStrategy.","commonSituations":"Trying to inspect or pass the mesh to other components right after constructing the strategy; logging/mesh introspection utilities that run pre-setup; using tensor-parallel APIs that expect a mesh before Lightning created it.","solutions":["Move any device_mesh access to inside the launched function, after fabric.setup / setup_environment has run","If you need a custom mesh, construct and pass your own DeviceMesh to the strategy rather than reading it early","Gate mesh-dependent code on mesh availability (check strategy._device_mesh or catch the error) during early phases"],"exampleFix":"# before\nstrategy = ModelParallelStrategy()\nmesh = strategy.device_mesh  # RuntimeError: not initialized yet\nfabric = Fabric(strategy=strategy)\n\n# after\nstrategy = ModelParallelStrategy()\nfabric = Fabric(strategy=strategy)\nwith fabric.init_module():\n    model = MyModel()\n# mesh is available after setup:\nmesh = strategy.device_mesh","handlingStrategy":"type-guard","validationCode":"if strategy._device_mesh is None:  # not yet set up\n    fabric.setup_environment()  # or wait until fabric.setup()/run()","typeGuard":"def mesh_ready(strategy) -> bool:\n    return getattr(strategy, \"_device_mesh\", None) is not None","tryCatchPattern":"try:\n    mesh = strategy.device_mesh\nexcept RuntimeError:\n    mesh = None  # defer mesh-dependent work until after setup","preventionTips":["Only access strategy resources inside the launched function after fabric setup","Pass an explicit DeviceMesh if you need it early"],"tags":["pytorch-lightning","model-parallel","device-mesh","initialization-order"],"backgroundTag":"premature-resource-access-before-setup","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}