hiyouga/LlamaFactory · error · ImportError

hyper_parallel is not installed. Please install it with `pip

Error message

hyper_parallel is not installed. Please install it with `pip install hyper_parallel`.

What it means

Error "hyper_parallel is not installed. Please install it with `pip install hyper_parallel`." thrown in hiyouga/LlamaFactory.

Source

Thrown at src/llamafactory/train/hyper_parallel/workflow.py:48

if TYPE_CHECKING:
    from transformers import Seq2SeqTrainingArguments, TrainerCallback

    from ...hparams import DataArguments, FinetuningArguments, GeneratingArguments, ModelArguments


logger = get_logger(__name__)


def _prepare_hp_args(finetuning_args: "FinetuningArguments", model_args: "ModelArguments"):
    r"""Load HyperParallel arguments and apply LlamaFactory-side overrides.

    When activation optimization is enabled, skip native gradient checkpointing
    so HP can install its own via ``setup_activation_optimization``.
    """
    if not is_hyper_parallel_available():
        raise ImportError("hyper_parallel is not installed. Please install it with `pip install hyper_parallel`.")

    from hyper_parallel.integration.llamafactory import HyperParallelArguments  # pylint: disable=C0415

    hp_args = HyperParallelArguments.from_finetuning_args(finetuning_args)

    if getattr(hp_args, "cp_size", None) != finetuning_args.hyper_parallel_cp_size:
        setattr(hp_args, "cp_size", finetuning_args.hyper_parallel_cp_size)

    if hp_args.activation_mode != "none":
        model_args.disable_gradient_checkpointing = True
    return hp_args


def run_pt(
    model_args: "ModelArguments",
    data_args: "DataArguments",
    training_args: "Seq2SeqTrainingArguments",
    finetuning_args: "FinetuningArguments",

View on GitHub (pinned to f28afaf635)

Solutions

1. Install the hyper_parallel package: pip install hyper_parallel.

Example fix

pip install hyper_parallel

When it happens

Trigger: Thrown at src/llamafactory/train/hyper_parallel/workflow.py:48 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/b810a768fe54535c. Report an issue: GitHub.