hiyouga/LlamaFactory · critical · RuntimeError

DeepSpeed ZeRO-3 model-loading bootstrap failed: transformer

Error message

DeepSpeed ZeRO-3 model-loading bootstrap failed: transformers still reports zero3 disabled after constructing HfDeepSpeedConfig. This usually means the runtime is using a different transformers installation than expected, or the DeepSpeed global state was not established correctly.

What it means

After building the accelerate DeepSpeedPlugin with zero3_init_flag=True and setting the weakref, the code asserts transformers' global is_deepspeed_zero3_enabled() is now True. If transformers still reports ZeRO-3 disabled, the bootstrap failed — typically a mismatched/duplicate transformers install or DeepSpeed global state never established (e.g. a second import path or an already-set HfDeepSpeedConfig).

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/deepspeed_utils.py:137

    zero_stage = ds_config.get("zero_optimization", {}).get("stage")
    if zero_stage != 3:
        return None

    # ZeRO-3 model loading needs concrete fp16/bf16 flags, not "auto".
    mixed_precision = infer_deepspeed_mixed_precision(ds_config)

    plugin = DeepSpeedPlugin(hf_ds_config=ds_config, zero3_init_flag=True)

    if not plugin.hf_ds_config.is_zero3():
        return None

    # Reuse the same precision inference rule as the training-time DeepSpeed path
    # so both model-loading and engine setup stay aligned.
    plugin.set_mixed_precision(mixed_precision)
    plugin.set_deepspeed_weakref()

    if not _hf_is_deepspeed_zero3_enabled():
        raise RuntimeError(
            "DeepSpeed ZeRO-3 model-loading bootstrap failed: transformers still reports zero3 disabled "
            "after constructing HfDeepSpeedConfig. This usually means the runtime is using a different transformers "
            "installation than expected, or the DeepSpeed global state was not established correctly."
        )
    return plugin


def teardown_deepspeed_zero3_model_loading(plugin) -> None:
    if plugin is not None:
        _unset_hf_deepspeed_config()

View on GitHub (pinned to f28afaf635)

Solutions

1. Verify a single transformers: `python -c "import transformers; print(transformers.__file__, transformers.__version__)"` and remove duplicates.
2. Reinstall the stack consistently in one environment: `pip install -U transformers accelerate deepspeed` (or `uv sync`).
3. Restart the process — this bootstrap must run once, before model creation; avoid re-running it after teardown in the same process.
4. Check deepspeed is importable and versions are compatible (deepspeed>=0.9.x with recent transformers).
5. If it persists, disable ZeRO-3 init (zero3_init_flag) so model loading uses the normal path.

Example fix

# before (mixed envs)
pip install transformers==4.46.0  # while an editable copy shadows it

# after
git clone ... && pip install -e .  # one canonical env
python -c "import transformers, accelerate, deepspeed; print(transformers.__file__, accelerate.__file__)"  # same env prefix

import transformers, accelerate, deepspeed, os
roots = {os.path.dirname(m.__file__.rsplit('/', 1)[0]) for m in (transformers, accelerate, deepspeed)}
print(transformers.__version__, transformers.__file__)
assert len(roots) <= 2, f'mixed site-packages: {roots}'  # tolerate one shared prefix

def transformers_env_clean() -> bool:
    """True when a single transformers install is importable and deepspeed loads."""
    import importlib.util
    specs = importlib.util.find_spec('transformers')
    return specs is not None and deepspeed_is_importable()

try:
    plugin = setup_deepspeed_zero3_model_loading()
except RuntimeError as e:
    if 'ZeRO-3 model-loading bootstrap failed' in str(e):
        logger.warning('falling back to non-zero3-init loading; fix env')
        plugin = None
    else:
        raise

When it happens

Trigger: Multiple transformers versions on sys.path (editable install vs site-packages), a conda/pip mix where accelerate binds one and llamafactory another; HfDeepSpeedConfig global already torn down or never set because of import order; running inside a process that reset DeepSpeed state.

Common situations: Docker images with layered pip installs; `pip install -e .` over a released transformers; notebooks that re-initialize distributed state; mismatched accelerate/transformers/deepspeed versions after a partial upgrade.

Related errors

• DeepSpeed config_file is required in dist_config
• Megatron Bridge is incompatible with DeepSpeed.
• Please use `FORCE_TORCHRUN=1` to launch DeepSpeed training.
• Layer-wise BAdam only supports DeepSpeed ZeRO-3 training.
• KTransformers is incompatible with DeepSpeed ZeRO-3.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/03d0e7b67b2b893d. Report an issue: GitHub.