hiyouga/LlamaFactory · error · ImportError

hyper_parallel is not installed. Please install it with `pip

Error message

hyper_parallel is not installed. Please install it with `pip install hyper_parallel`.

What it means

In `_training_function` (src/llamafactory/train/tuner.py:94), enabling `use_hyper_parallel` for pt/sft first checks availability via `is_hyper_parallel_available()`; if the optional `hyper_parallel` package is absent, ImportError with install instructions is raised before any training starts.

Source

Thrown at src/llamafactory/train/tuner.py:94

        callbacks.append(PissaConvertCallback())

    if finetuning_args.use_swanlab:
        callbacks.append(get_swanlab_callback(finetuning_args))

    if finetuning_args.early_stopping_steps is not None:
        callbacks.append(EarlyStoppingCallback(early_stopping_patience=finetuning_args.early_stopping_steps))

    if getattr(training_args, "enable_torch_profiler", False):
        callbacks.append(TorchProfilerCallback(training_args))

    if getattr(training_args, "profile_modules", None):
        callbacks.append(ModuleProfilerCallback(training_args.profile_modules))

    callbacks.append(ReporterCallback(model_args, data_args, finetuning_args, generating_args))  # add to last

    if finetuning_args.stage in ["pt", "sft"] and finetuning_args.use_hyper_parallel:
        if not is_hyper_parallel_available():
            raise ImportError("hyper_parallel is not installed. Please install it with `pip install hyper_parallel`.")
        if finetuning_args.stage == "pt":
            from .hyper_parallel import run_pt as run_pt_hp

            run_pt_hp(model_args, data_args, training_args, finetuning_args, callbacks)
        else:
            from .hyper_parallel import run_sft as run_sft_hp

            run_sft_hp(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)

    elif finetuning_args.stage in ["pt", "sft"] and finetuning_args.use_megatron_bridge:
        if not is_megatron_bridge_available():
            raise ImportError(
                "megatron-bridge is not installed. "
                "Please install it with `pip install --no-build-isolation megatron-bridge`."
            )
        mb_args = finetuning_args.megatron_bridge_args
        if mb_args is None:
            raise ValueError("Megatron Bridge arguments are missing. Please set USE_MEGATRON_BRIDGE=1.")

View on GitHub (pinned to f28afaf635)

Solutions

Install the package: `pip install hyper_parallel`.
Or install LlamaFactory with the extras that include it, if your fork publishes them.
If FSDP2 HP is not needed, set `use_hyper_parallel: false` and use the standard trainer.

Example fix

# shell
# before: run fails with ImportError
pip install hyper_parallel  # then rerun training

Defensive patterns

Strategy: validation

Validate before calling

from importlib.util import find_spec
def hyper_parallel_ready():
    return find_spec("hyper_parallel") is not None

Prevention

Check optional extras with find_spec before setting their feature flags.
Bake optional dependencies into the Dockerfile used for distributed training.

When it happens

Trigger: Config with `use_hyper_parallel: true` on an environment where `pip install hyper_parallel` was never run (hyper_parallel is an optional extra, not a core dependency).

Common situations: Copying a HyperParallel FSDP2 example YAML into a base install; CI images built from the minimal dependency set.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/72b0c77211f9efac. Report an issue: GitHub.