Lightning-AI/pytorch-lightning · error · RuntimeError

You are calling the method `{type(self._original_module).__n

Error message

You are calling the method `{type(self._original_module).__name__}.{name}()` from outside the model. To avoid issues with the currently selected strategy, explicitly mark it as a forward method with `fabric_model.mark_forward_method({name!r})` after `fabric.setup()`.

What it means

When a non-forward method of a wrapped FabricModule is called from outside the module, forward hooks are temporarily registered to detect whether the call went through the model's forward (and hence the strategy, e.g. DDP gradient sync). If no forward was triggered, Lightning raises this RuntimeError because calling e.g. model.generate() directly would bypass the strategy and silently break gradient synchronization.

Source

Thrown at src/lightning/fabric/wrappers.py:219

    def _wrap_method_with_module_call_tracker(self, method: Callable, name: str) -> Callable:
        """Tracks whether any submodule in ``self._original_module`` was called during the execution of ``method`` by
        registering forward hooks on all submodules."""
        module_called = False

        def hook(*_: Any, **__: Any) -> None:
            nonlocal module_called
            module_called = True

        @wraps(method)
        def _wrapped_method(*args: Any, **kwargs: Any) -> Any:
            handles = []
            for module in self._original_module.modules():
                handles.append(module.register_forward_hook(hook))

            output = method(*args, **kwargs)

            if module_called:
                raise RuntimeError(
                    f"You are calling the method `{type(self._original_module).__name__}.{name}()` from outside the"
                    " model. To avoid issues with the currently selected strategy, explicitly mark it as a"
                    f" forward method with `fabric_model.mark_forward_method({name!r})` after `fabric.setup()`."
                )
            for handle in handles:
                handle.remove()
            return output

        return _wrapped_method

    def _register_backward_hook(self, tensor: Tensor) -> Tensor:
        if not tensor.requires_grad:
            return tensor

        strategy_requires = is_overridden("backward", self._strategy, parent=Strategy)
        precision_requires = any(
            is_overridden(method, self._strategy.precision, parent=Precision)
            for method in ("pre_backward", "backward", "post_backward")

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Mark the method right after setup: fabric_module.mark_forward_method('generate')
2. Or move the logic into the module's forward() so it is routed through the strategy
3. Or call the method on the unwrapped module only when no gradients/sync are needed (inference-only)

Example fix

# before
model = fabric.setup(model)
out = model.generate(input_ids, do_sample=True)  # RuntimeError

# after
model = fabric.setup(model)
model.mark_forward_method('generate')
out = model.generate(input_ids, do_sample=True)

name = 'generate'
if name not in getattr(fabric_module, '_forward_methods', set()) and callable(getattr(fabric_module, name, None)):
    fabric_module.mark_forward_method(name)

When it happens

Trigger: Calling fabric_module.generate(...) (a method not in _forward_methods) directly in your training loop after fabric.setup(), under a strategy like DDP that relies on forward-based hooks/sync.

Common situations: Using HF transformers-style model.generate() or custom methods (encode, decode) on a Fabric-wrapped module under DDP/FSDP without marking them first; refactoring a single-GPU script to multi-GPU Fabric; new model methods added after setup.

Related errors

• You need to set up the model first before you can call `fabr
• To use Fabric with more than one device, you must call `.lau
• `setup_optimizers` requires at least one optimizer as input.
• An optimizer should be passed only once to the `setup_optimi
• The optimizer has references to the model's meta-device para

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/af391673c4dd180e. Report an issue: GitHub.