hiyouga/LlamaFactory · error · ImportError

megatron-bridge is not installed. Please install it with `pi

Error message

megatron-bridge is not installed. Please install it with `pip install --no-build-isolation megatron-bridge` or use the NeMo Framework container.

What it means

Error "megatron-bridge is not installed. Please install it with `pip install --no-build-isolation megatron-bridge` or use the NeMo Framework container." thrown in hiyouga/LlamaFactory.

Source

Thrown at src/llamafactory/train/megatron_bridge/workflow.py:98

        os.makedirs(sync_dir, exist_ok=True)
        if os.path.isfile(done_file):
            os.remove(done_file)
        work()
        with open(done_file, "w", encoding="utf-8") as f:
            f.write("done")

    if dist_ready:
        dist.barrier()
    elif not is_main:
        while True:
            if os.path.isfile(done_file) and os.path.getmtime(done_file) >= wait_start - 1.0:
                break
            time.sleep(0.5)


def _check_backend_available() -> None:
    if not is_megatron_bridge_available():
        raise ImportError(
            "megatron-bridge is not installed. "
            "Please install it with `pip install --no-build-isolation megatron-bridge` "
            "or use the NeMo Framework container."
        )
    _patch_dataset_helper_compilation()
    _patch_dist_checkpoint_preload()


def _patch_dist_checkpoint_preload() -> None:
    r"""Use blocking GPU->CPU copies when saving distributed checkpoints.

    Megatron's default ``non_blocking=True`` preload can raise ``cudaErrorInvalidValue``
    on some GPUs (e.g. V100) when saving distributed optimizer shards, because pinned
    host memory allocation or async D2H transfer may fail under memory pressure.
    """
    from megatron.core.dist_checkpointing.strategies import filesystem_async

    if getattr(filesystem_async.FileSystemWriterAsync.preload_tensors, "_llamafactory_patched", False):

View on GitHub (pinned to f28afaf635)

Solutions

1. Install megatron-bridge: pip install --no-build-isolation megatron-bridge.
2. Alternatively run inside the NVIDIA NeMo Framework container which ships megatron-bridge.

Example fix

pip install --no-build-isolation megatron-bridge

When it happens

Trigger: Thrown at src/llamafactory/train/megatron_bridge/workflow.py:98 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/a699bc6021cfcb51. Report an issue: GitHub.