huggingface/transformers · error · RuntimeError

You are using `from_pretrained` with a meta device context m

Error message

You are using `from_pretrained` with a meta device context manager or `torch.set_default_device('meta')`.\nThis is an anti-pattern as `from_pretrained` wants to load existing weights.\nIf you want to initialize an empty model on the meta device, use the context manager or global device with `from_config`, or `ModelClass(config)`

What it means

Error "You are using `from_pretrained` with a meta device context manager or `torch.set_default_device('meta')`.\nThis is an anti-pattern as `from_pretrained` wants to load existing weights.\nIf you want to initialize an empty model on the meta device, use the context manager or global device with `from_config`, or `ModelClass(config)`" thrown in huggingface/transformers.

Source

Thrown at src/transformers/integrations/accelerate.py:101

    for tied_param in tied_params:
        tied_module_index = [i for i, (n, _) in enumerate(modules_to_treat) if tied_param.startswith(n + ".")][0]
        tied_module_name = modules_to_treat[tied_module_index][0]
        if tied_module_name not in tied_module_names:
            tied_module_names.append(tied_module_name)
            tied_modules.append(modules_to_treat[tied_module_index][1])
            module_size_with_ties += module_sizes[tied_module_name]

    return module_size_with_ties, tied_module_names, tied_modules


def check_and_set_device_map(device_map: "torch.device | int | str | dict | None") -> dict | str | None:
    from ..modeling_utils import get_torch_context_manager_or_global_device

    # Potentially detect context manager or global device, and use it (only if no device_map was provided)
    if device_map is None and not is_deepspeed_zero3_enabled():
        device_in_context = get_torch_context_manager_or_global_device()
        if device_in_context == torch.device("meta"):
            raise RuntimeError(
                "You are using `from_pretrained` with a meta device context manager or `torch.set_default_device('meta')`.\n"
                "This is an anti-pattern as `from_pretrained` wants to load existing weights.\nIf you want to initialize an "
                "empty model on the meta device, use the context manager or global device with `from_config`, or `ModelClass(config)`"
            )
        device_map = device_in_context

    # change device_map into a map if we passed an int, a str or a torch.device
    if isinstance(device_map, torch.device):
        device_map = {"": device_map}
    elif isinstance(device_map, str) and device_map not in ["auto", "balanced", "balanced_low_0", "sequential"]:
        try:
            if device_map == "cuda":
                # setting to the local rank
                local_rank = int(os.environ.get("LOCAL_RANK", 0))
                device_map = f"cuda:{local_rank}"
            device_map = {"": torch.device(device_map)}
        except RuntimeError:
            raise ValueError(

View on GitHub (pinned to a597f97485)

Solutions

1. Do not wrap `from_pretrained` in a meta-device context or `torch.set_default_device('meta')`.
2. Use `from_config` or `ModelClass(config)` to create an empty meta-device model.

When it happens

Trigger: Raised in from_pretrained device handling when a meta-device context or torch.set_default_device('meta') is active.

Common situations: Loading pretrained weights inside init_empty_weights or with default device set to meta; weights cannot load onto meta tensors.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/badf310e1d4b1c94. Report an issue: GitHub.