huggingface/transformers · error · ValueError

The current `device_map` had weights offloaded to the disk,

Error message

The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal weight format different than the one saved (i.e. most MoE models). Please provide an `offload_folder` for them in `from_pretrained`.

What it means

Error "The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal weight format different than the one saved (i.e. most MoE models). Please provide an `offload_folder` for them in `from_pretrained`." thrown in huggingface/transformers.

Source

Thrown at src/transformers/integrations/accelerate.py:503

        # Tie weights which are both disk offloaded
        all_tied_weights_keys = getattr(model, "all_tied_weights_keys", {})
        for target_param_name, source_param_name in all_tied_weights_keys.items():
            if source_param_name in disk_offload_index and target_param_name not in disk_offload_index:
                disk_offload_index[target_param_name] = disk_offload_index[source_param_name]

    # In this case we will resave every offloaded weight
    else:
        disk_offload_index = {}

    return disk_offload_index


def offload_weight(weight: torch.Tensor, weight_name: str, offload_folder: str | None, offload_index: dict) -> dict:
    """Write `weight` to disk inside `offload_folder`, and update `offload_index` accordingly. Everything is
    saved in `safetensors` format."""

    if offload_folder is None:
        raise ValueError(
            "The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either "
            "because the weights are not in `safetensors` format, or because the model uses an internal weight format "
            "different than the one saved (i.e. most MoE models). Please provide an `offload_folder` for them in "
            "`from_pretrained`."
        )
    # Write the weight to disk
    safetensor_file = os.path.join(offload_folder, f"{weight_name}.safetensors")
    save_file({weight_name: weight}, safetensor_file)
    # Update the offloading index
    str_dtype = str(weight.dtype).replace("torch.", "")
    offload_index[weight_name] = {"safetensors_file": safetensor_file, "weight_name": weight_name, "dtype": str_dtype}
    return offload_index


def load_offloaded_parameter(model: "PreTrainedModel", param_name: str) -> torch.Tensor:
    """Load `param_name` from disk, if it was offloaded due to the device_map, and thus lives as a meta parameter
    inside `model`.
    This is needed when resaving a model, when some parameters were offloaded (we need to load them from disk, to

View on GitHub (pinned to a597f97485)

Solutions

1. Pass an `offload_folder` to `from_pretrained` for disk-offloaded weights.
2. Re-save the checkpoint in safetensors format.

When it happens

Trigger: Raised during from_pretrained dispatch when some weights must be offloaded to disk but no offload_folder was provided.

Common situations: device_map with disk offload on non-safetensors or MoE checkpoints without specifying offload_folder.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/38888d2d01dfde77. Report an issue: GitHub.