{"record":{"id":"75efde825de8eed0","repo":"Lightning-AI/pytorch-lightning","slug":"blocking-backward-sync-is-only-possible-if-the-mod","errorCode":null,"errorMessage":"Blocking backward sync is only possible if the module passed to `{self.__class__.__name__}.no_backward_sync` is wrapped in `DistributedDataParallel`. Got: {module.__class__.__name__}.","messagePattern":"Blocking backward sync is only possible if the module passed to `(.+?)\\.no_backward_sync` is wrapped in `DistributedDataParallel`\\. Got: (.+?)\\.","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"src/lightning/fabric/strategies/ddp.py","lineNumber":268,"sourceCode":"            self.cluster_environment.set_world_size(self.num_nodes * self.num_processes)\n        # `LightningEnvironment.set_global_rank` will do this too, but we cannot rely on that implementation detail\n        # additionally, for some implementations, the setter is a no-op, so it's safer to access the getter\n        rank_zero_only.rank = utils_rank_zero_only.rank = self.global_rank\n\n    def _determine_ddp_device_ids(self) -> Optional[list[int]]:\n        return None if self.root_device.type == \"cpu\" else [self.root_device.index]\n\n\nclass _DDPBackwardSyncControl(_BackwardSyncControl):\n    @override\n    def no_backward_sync(self, module: Module, enabled: bool) -> AbstractContextManager:\n        \"\"\"Blocks gradient synchronization inside the :class:`~torch.nn.parallel.distributed.DistributedDataParallel`\n        wrapper.\"\"\"\n        if not enabled:\n            return nullcontext()\n\n        if not isinstance(module, DistributedDataParallel):\n            raise TypeError(\n                \"Blocking backward sync is only possible if the module passed to\"\n                f\" `{self.__class__.__name__}.no_backward_sync` is wrapped in `DistributedDataParallel`.\"\n                f\" Got: {module.__class__.__name__}.\"\n            )\n        return module.no_sync()\n","sourceCodeStart":250,"sourceCodeEnd":274,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/strategies/ddp.py#L250-L274","documentation":"DDPStrategy.no_backward_sync(enabled=True) returns module.no_sync(), which only exists on torch's DistributedDataParallel wrapper. If the module you pass is a plain nn.Module (or the strategy hasn't wrapped it yet / you unwrapped it), a TypeError is raised telling you to pass the DDP-wrapped module.","triggerScenarios":"Calling fabric.strategy.no_backward_sync(module) or using the fabric.no_backward_sync context with gradient accumulation while the model is a raw nn.Module — e.g. before strategy.setup(module), after strategy.unwrap_module(model), or with a non-DDP strategy fallback.","commonSituations":"Gradient-accumulation loops that skip the no-sync context on the last batch; passing fabric.model after it was unwrapped for logging; calling no_backward_sync before setup wrapped the module in DistributedDataParallel.","solutions":["Pass the wrapped module — use fabric.no_backward_sync(...) as a context manager which uses the strategy's wrapped model, or ensure strategy.setup(module) ran first","Only call no_backward_sync when enabled=True is needed on accumulation steps; the method returns nullcontext() when enabled=False, so gating it avoids the check entirely","Ensure the strategy actually is DDPStrategy (single-device or deepspeed strategies don't produce a DDP wrapper)"],"exampleFix":"# before\nwith fabric.strategy.no_backward_sync(raw_module):  # raw nn.Module\n    ...\n# after\nwith fabric.no_backward_sync(is_first_batch):  # uses wrapped fabric.model\n    ...","handlingStrategy":"type-guard","validationCode":"from torch.nn.parallel import DistributedDataParallel as DDP\nif enabled and not isinstance(module, DDP):\n    module = fabric.strategy.model if isinstance(fabric.strategy.model, DDP) else module\n    # or skip no-sync on this step","typeGuard":"from torch.nn.parallel import DistributedDataParallel as DDP\nfrom contextlib import nullcontext\n\ndef safe_no_backward_sync(strategy, module, enabled: bool):\n    if not enabled or not isinstance(module, DDP):\n        return nullcontext()\n    return module.no_sync()","tryCatchPattern":null,"preventionTips":["Use fabric.no_backward_sync(...) which operates on the wrapped model","Always run strategy.setup/fabric.setup before using sync control APIs"],"tags":["ddp","gradient-accumulation","type-mismatch","pytorch-lightning"],"backgroundTag":"wrong-argument-type","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}