huggingface/transformers · error · ValueError

{param_name} is on the meta device because it was offloaded,

Error message

{param_name} is on the meta device because it was offloaded, but we could not find the corresponding hook for it

What it means

Error "{param_name} is on the meta device because it was offloaded, but we could not find the corresponding hook for it" thrown in huggingface/transformers.

Source

Thrown at src/transformers/integrations/accelerate.py:534

def load_offloaded_parameter(model: "PreTrainedModel", param_name: str) -> torch.Tensor:
    """Load `param_name` from disk, if it was offloaded due to the device_map, and thus lives as a meta parameter
    inside `model`.
    This is needed when resaving a model, when some parameters were offloaded (we need to load them from disk, to
    then resave them to disk in the correct shard...)."""
    # Start from the most inner module, and try to find the hook that was used for offloading the param
    module_parts = param_name.split(".")
    modules_to_check = [".".join(module_parts[:-idx]) for idx in range(1, len(module_parts))] + [""]
    for parent_name in modules_to_check:
        parent = model.get_submodule(parent_name)
        if hasattr(parent, "_hf_hook"):
            weights_map = parent._hf_hook.weights_map
            truncated_param_name = param_name.replace(f"{parent_name}." if parent_name != "" else parent_name, "")
            break
    # If we did not break the loop, something is wrong
    else:
        raise ValueError(
            f"{param_name} is on the meta device because it was offloaded, but we could not find "
            "the corresponding hook for it"
        )

    # This call loads it from disk
    tensor = weights_map[truncated_param_name]
    return tensor


def _init_infer_auto_device_map(
    model: nn.Module,
    max_memory: dict[int | str, int | str] | None = None,
    no_split_module_classes: set[str] | None = None,
    tied_parameters: list[list[str]] | None = None,
    hf_quantizer: "HfQuantizer | None" = None,
) -> tuple[
    list[int | str],
    dict[int | str, int | str],

View on GitHub (pinned to a597f97485)

Solutions

1. Re-dispatch the model with `dispatch_model` so hooks are attached for offloaded params.
2. Avoid manually moving offloaded parameters to the meta device.

When it happens

Trigger: Raised when a parameter is on the meta device due to offload but its align_devices hook cannot be found.

Common situations: Model dispatched with hooks partially removed or manually moved after accelerate dispatch, breaking offloaded weight lookup.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/2817b0bde8b1d704. Report an issue: GitHub.