hiyouga/LlamaFactory · error · ValueError

DeepSpeed config_file is required in dist_config

Error message

DeepSpeed config_file is required in dist_config

What it means

setup_deepspeed_zero3_model_loading() needs the path to a DeepSpeed JSON config to construct the accelerate DeepSpeedPlugin with zero3_init. It reads it from the registered distributed config dict; if config_file is missing, empty, or no dist config was registered at all, it raises this ValueError.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/deepspeed_utils.py:100

    try:
        from transformers.integrations import unset_hf_deepspeed_config
    except ImportError:
        from transformers.deepspeed import unset_hf_deepspeed_config

    unset_hf_deepspeed_config()


def _load_deepspeed_config(config_file: str) -> dict[str, Any]:
    with open(config_file, encoding="utf-8") as f:
        return json.load(f)


def setup_deepspeed_zero3_model_loading():
    """Enable ZeRO-3-aware model loading for the registered backend config."""
    dist_config = _registered_dist_config
    config_file = dist_config.get("config_file") if dist_config is not None else None
    if not config_file:
        raise ValueError("DeepSpeed config_file is required in dist_config")

    from accelerate.utils import DeepSpeedPlugin

    try:
        from transformers.integrations import is_deepspeed_zero3_enabled as _hf_is_deepspeed_zero3_enabled
    except ImportError:
        from transformers.deepspeed import is_deepspeed_zero3_enabled as _hf_is_deepspeed_zero3_enabled

    # DeepSpeed configs often use "auto" placeholders that only make sense once
    # we know the current runtime batch settings and precision mode.
    ds_config = deepcopy(_load_deepspeed_config(config_file))
    if "gradient_accumulation_steps" not in ds_config or ds_config["gradient_accumulation_steps"] == "auto":
        ds_config["gradient_accumulation_steps"] = 1
    if "train_micro_batch_size_per_gpu" not in ds_config or ds_config["train_micro_batch_size_per_gpu"] == "auto":
        ds_config["train_micro_batch_size_per_gpu"] = 1
    if ds_config.get("train_batch_size") == "auto":
        ds_config.pop("train_batch_size")

View on GitHub (pinned to f28afaf635)

Solutions

1. Set the 'config_file' key in the registered dist_config to a valid DeepSpeed JSON path before calling setup.
2. In YAML configs, point the deepspeed option at the JSON file so registration carries it.
3. Check the file exists and is readable JSON (a bad path fails later in _load_deepspeed_config).
4. If you do not want ZeRO-3 init loading, skip calling this function at all.

Example fix

# before
dist_config = {"engine": "deepspeed"}  # no config_file
setup_deepspeed_zero3_model_loading()

# after
dist_config = {"engine": "deepspeed", "config_file": "ds_zero3.json"}
setup_deepspeed_zero3_model_loading()

import os
cfg_file = dist_config.get('config_file') if dist_config else None
assert cfg_file and os.path.isfile(cfg_file), 'DeepSpeed config_file missing or not a file'

def has_deepspeed_config(dist_config: dict | None) -> bool:
    """True when dist_config carries an existing config_file path."""
    f = (dist_config or {}).get('config_file')
    return bool(f) and os.path.isfile(f)

try:
    plugin = setup_deepspeed_zero3_model_loading()
except ValueError as e:
    if 'config_file is required' in str(e):
        raise SystemExit('set deepspeed: ds_zero3.json in the YAML') from None
    raise

When it happens

Trigger: Calling setup_deepspeed_zero3_model_loading() before registering the dist config, or registering a dist config without a 'config_file' key (e.g. only engine/args set programmatically).

Common situations: Programmatic use of the v1 API where DeepSpeed is enabled via arguments rather than a YAML that carries deepspeed: path; typos in the config key; forgetting that ZeRO-3 init loading requires an explicit config file.

Related errors

• Megatron Bridge is incompatible with DeepSpeed.
• DeepSpeed only supports bf16 mixed precision for now, fp16 i
• DeepSpeed ZeRO-3 model-loading bootstrap failed: transformer
• DeepSpeed config_file is required.
• `tensor_model_parallel_size` must be >= 1.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/d3e238d27f51c70e. Report an issue: GitHub.