hiyouga/LlamaFactory · error · ValueError

DeepSpeed config_file is required in dist_config

Error message

DeepSpeed config_file is required in dist_config

What it means

The v1 DeepSpeed plugin wraps accelerate's DeepSpeedPlugin, which requires a DeepSpeed JSON config file. The wrapper reads dist_config['config_file'] and aborts if missing or empty, because accelerate cannot construct the plugin without it. Inline dict configs are not accepted by this wrapper.

Source

Thrown at src/llamafactory/v1/plugins/trainer_plugins/distributed/deepspeed.py:60

}


class DeepSpeedEngine:
    """DeepSpeed integration using accelerate's built-in capabilities.

    This replaces the manual DeepSpeedConfigHelper / DeepSpeedEngine approach
    with accelerate's Accelerator + DeepSpeedPlugin, which handles:
    - Config syncing (auto values, batch size, lr, etc.)
    - deepspeed.initialize() call
    - Optimizer / LR scheduler wrapping
    - Backward + gradient accumulation boundary
    - ZeRO-3 parameter gathering for saving
    """

    def __init__(self, dist_config: dict[str, Any], num_micro_batch: int = 1, micro_batch_size: int = 1):
        config_file = dist_config.get("config_file")
        if not config_file:
            raise ValueError("DeepSpeed config_file is required in dist_config")

        ds_plugin = DeepSpeedPlugin(hf_ds_config=config_file)
        ds_plugin.set_mixed_precision(infer_deepspeed_mixed_precision(ds_plugin.deepspeed_config))

        self.accelerator = Accelerator(
            deepspeed_plugin=ds_plugin,
            gradient_accumulation_steps=num_micro_batch,
        )

        # Resolve "auto" for train_micro_batch_size_per_gpu so that
        # accelerate.prepare() does not require a DataLoader to infer it.
        ds_config = self.accelerator.state.deepspeed_plugin.deepspeed_config
        if ds_config.get("train_micro_batch_size_per_gpu") in (None, "auto"):
            ds_config["train_micro_batch_size_per_gpu"] = micro_batch_size

        logger.info_rank0(f"DeepSpeedEngine initialized with config: {config_file}")

    def shard_model(self, model: HFModel) -> "DeepSpeedEngine":

View on GitHub (pinned to f28afaf635)

Solutions

1. Create a DeepSpeed JSON config (e.g. examples/deepspeed/ds_z2_config.json) and set dist_config['config_file'] to its path
2. Verify the path exists and is readable before launching
3. Do not pass the DeepSpeed settings as an inline dict; only the file reference is supported here

Example fix

# before
dist_config = {"zero_stage": 2}

# after
dist_config = {"config_file": "examples/deepspeed/ds_z2_config.json"}

cfg_file = dist_config.get("config_file")
assert cfg_file and os.path.isfile(cfg_file), f"dist_config.config_file must reference an existing DeepSpeed JSON, got {cfg_file!r}"

When it happens

Trigger: Initializing the DeepSpeed distributed backend with a dist_config dict that has no 'config_file' key, or where config_file is an empty string / None.

Common situations: User writes ds_config settings inline in dist_config instead of referencing a ds_z2_config.json file; path variable resolves to empty string due to env or YAML interpolation; migrating from v0 where config was resolved differently.

Related errors

• Megatron Bridge is incompatible with DeepSpeed.
• Please use `FORCE_TORCHRUN=1` to launch DeepSpeed training.
• Layer-wise BAdam only supports DeepSpeed ZeRO-3 training.
• Context parallelism currently requires `dist_config.name: fs
• DeepSpeed only supports bf16 mixed precision for now, fp16 i

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/3f3c370634ad0e9e. Report an issue: GitHub.