huggingface/transformers · error · ValueError

DeepSpeed Zero-3 is not compatible with passing a `device_ma

Error message

DeepSpeed Zero-3 is not compatible with passing a `device_map`.

What it means

Error "DeepSpeed Zero-3 is not compatible with passing a `device_map`." thrown in huggingface/transformers.

Source

Thrown at src/transformers/integrations/accelerate.py:133

                local_rank = int(os.environ.get("LOCAL_RANK", 0))
                device_map = f"cuda:{local_rank}"
            device_map = {"": torch.device(device_map)}
        except RuntimeError:
            raise ValueError(
                "When passing device_map as a string, the value needs to be a device name (e.g. cpu, cuda:0) or "
                f"'auto', 'balanced', 'balanced_low_0', 'sequential' but found {device_map}."
            )
    elif isinstance(device_map, int):
        if device_map < 0:
            raise ValueError(
                "You can't pass device_map as a negative int. If you want to put the model on the cpu, pass device_map = 'cpu' "
            )
        else:
            device_map = {"": device_map}

    if device_map is not None:
        if is_deepspeed_zero3_enabled():
            raise ValueError("DeepSpeed Zero-3 is not compatible with passing a `device_map`.")
        if not is_accelerate_available():
            raise ValueError(
                "Using a `device_map`, `tp_plan`, `torch.device` context manager or setting `torch.set_default_device(device)` "
                "requires `accelerate`. You can install it with `pip install accelerate`"
            )
    return device_map


def compute_module_sizes(
    model: "PreTrainedModel",
    hf_quantizer: "HfQuantizer | None" = None,
    buffers_only: bool = False,
    only_modules: bool = True,
) -> tuple[dict[str, int], dict[str, int]]:
    """
    Compute the size of each submodule of a given model (in bytes).
    Returns a tuple of 2 dicts, the first one containing a mapping of all the modules and the corresponding size
    in bytes, and the 2nd one containing a mapping from all leaf modules (modules containing parameters, the end of

View on GitHub (pinned to a597f97485)

Solutions

1. Remove the `device_map` argument when using DeepSpeed Zero-3.
2. Let DeepSpeed manage device placement via its config.

When it happens

Trigger: Raised in from_pretrained when DeepSpeed ZeRO-3 is active and a device_map is also passed.

Common situations: Combining a ZeRO-3 DeepSpeed config with device_map in from_pretrained; the two placement mechanisms conflict.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/c735e6434ededddf. Report an issue: GitHub.