Lightning-AI/pytorch-lightning · error · TypeError

Materialization requires that the `{type(module).__name__}.r

Error message

Materialization requires that the `{type(module).__name__}.reset_parameters` method is implemented. This method is used to initialize any children parameters or buffers in this module.

What it means

When moving a module off the meta device, Lightning calls `to_empty` and then relies on `reset_parameters()` to re-initialize the newly allocated (empty) tensors. If the module (or the object you passed to `Fabric.setup`/`init_module`) does not implement `reset_parameters`, there is no way to initialize the parameters, so Lightning raises a TypeError. Custom modules created on the meta device must implement this method.

Source

Thrown at src/lightning/fabric/utilities/init.py:66

        types: Sequence,
        args: Sequence[Any] = (),
        kwargs: Optional[dict] = None,
    ) -> Any:
        kwargs = kwargs or {}
        if not self.enabled:
            return func(*args, **kwargs)
        if getattr(func, "__module__", None) == "torch.nn.init":
            if "tensor" in kwargs:
                return kwargs["tensor"]
            return args[0]
        return func(*args, **kwargs)


def _materialize(module: Module, device: _DEVICE) -> None:
    """Materialize a module."""
    module.to_empty(device=device, recurse=False)
    if not hasattr(module, "reset_parameters"):
        raise TypeError(
            f"Materialization requires that the `{type(module).__name__}.reset_parameters` method is implemented."
            " This method is used to initialize any children parameters or buffers in this module."
        )
    if callable(module.reset_parameters):
        module.reset_parameters()


def _materialize_meta_tensors(module: Module, device: _DEVICE) -> None:
    """Materialize all tensors in a given module."""
    for module in module.modules():
        if _has_meta_device_parameters_or_buffers(module, recurse=False):
            _materialize(module, device)


def _materialize_distributed_module(module: Module, device: torch.device) -> None:
    # Reference: https://github.com/pytorch/torchtitan/blob/main/docs/fsdp.md#meta-device-initialization
    # TODO: Introduce `Fabric.materialize(module)` to give user control when materialization should happen
    # TODO: Make `torchmetrics.Metric` compatible with the `to_empty()` + `reset_parameters()` semantics

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Implement `reset_parameters(self)` on your custom module that re-initializes parameters/buffers (e.g. call `nn.init` functions or children's `reset_parameters`)
2. If the weights come from a checkpoint, implement a no-op or loading `reset_parameters` and load weights after materialization
3. Avoid meta-device initialization (skip `init_module`/`to_empty`) if you cannot modify the module

Example fix

// before
class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.lin = nn.Linear(4, 4)

// after
class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.lin = nn.Linear(4, 4)

    def reset_parameters(self) -> None:
        self.lin.reset_parameters()

hasattr(model, "reset_parameters") and callable(model.reset_parameters)

def supports_materialization(m) -> bool:
    return isinstance(m, torch.nn.Module) and callable(getattr(m, "reset_parameters", None))

When it happens

Trigger: Using `Fabric`/`Trainer` with a device that materializes modules lazily (meta device init, `init_module`, or FSDP/`materialize_distributed_module`) where the user's custom `nn.Module` subclasses something without `reset_parameters` (e.g. a plain Module wrapper) and does not define it itself.

Common situations: Custom model classes used with `with fabric.init_module():` on GPU/meta-device workflows; wrapping pretrained models that lack `reset_parameters`; upgrading Lightning to versions where meta-device init became the default path.

Related errors

• The optimizer has references to the model's meta-device para
• Received multiple values for {', '.join(duplicated_plugin_ke
• Received both `precision={precision_input}` and `plugins={se
• accelerator set through both strategy class and accelerator
• precision set through both strategy class and plugins, choos

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/ac5b68eddc82d846. Report an issue: GitHub.