{"record":{"id":"c735e6434ededddf","repo":"huggingface/transformers","slug":"deepspeed-zero-3-is-not-compatible-with-passing-a","errorCode":null,"errorMessage":"DeepSpeed Zero-3 is not compatible with passing a `device_map`.","messagePattern":"DeepSpeed Zero-3 is not compatible with passing a `device_map`\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/integrations/accelerate.py","lineNumber":133,"sourceCode":"                local_rank = int(os.environ.get(\"LOCAL_RANK\", 0))\n                device_map = f\"cuda:{local_rank}\"\n            device_map = {\"\": torch.device(device_map)}\n        except RuntimeError:\n            raise ValueError(\n                \"When passing device_map as a string, the value needs to be a device name (e.g. cpu, cuda:0) or \"\n                f\"'auto', 'balanced', 'balanced_low_0', 'sequential' but found {device_map}.\"\n            )\n    elif isinstance(device_map, int):\n        if device_map < 0:\n            raise ValueError(\n                \"You can't pass device_map as a negative int. If you want to put the model on the cpu, pass device_map = 'cpu' \"\n            )\n        else:\n            device_map = {\"\": device_map}\n\n    if device_map is not None:\n        if is_deepspeed_zero3_enabled():\n            raise ValueError(\"DeepSpeed Zero-3 is not compatible with passing a `device_map`.\")\n        if not is_accelerate_available():\n            raise ValueError(\n                \"Using a `device_map`, `tp_plan`, `torch.device` context manager or setting `torch.set_default_device(device)` \"\n                \"requires `accelerate`. You can install it with `pip install accelerate`\"\n            )\n    return device_map\n\n\ndef compute_module_sizes(\n    model: \"PreTrainedModel\",\n    hf_quantizer: \"HfQuantizer | None\" = None,\n    buffers_only: bool = False,\n    only_modules: bool = True,\n) -> tuple[dict[str, int], dict[str, int]]:\n    \"\"\"\n    Compute the size of each submodule of a given model (in bytes).\n    Returns a tuple of 2 dicts, the first one containing a mapping of all the modules and the corresponding size\n    in bytes, and the 2nd one containing a mapping from all leaf modules (modules containing parameters, the end of","sourceCodeStart":115,"sourceCodeEnd":151,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/integrations/accelerate.py#L115-L151","documentation":"Error \"DeepSpeed Zero-3 is not compatible with passing a `device_map`.\" thrown in huggingface/transformers.","triggerScenarios":"Raised in from_pretrained when DeepSpeed ZeRO-3 is active and a device_map is also passed.","commonSituations":"Combining a ZeRO-3 DeepSpeed config with device_map in from_pretrained; the two placement mechanisms conflict.","solutions":["Remove the `device_map` argument when using DeepSpeed Zero-3.","Let DeepSpeed manage device placement via its config."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}