hiyouga/LlamaFactory · error · ValueError

Pipeline parallel size should be smaller than the number of

Error message

Pipeline parallel size should be smaller than the number of gpus.

What it means

The mcore_adapter (Megatron-Core adapter) workflow is optional and imported lazily; at module import time workflow.py:43 checks is_mcore_adapter_available() and raises ImportError with the pip package name if the adapter is missing, because all subsequent imports (mcore_adapter.models, trainer, DPOConfig) depend on it.

Source

Thrown at scripts/vllm_infer.py:79

    max_new_tokens: int = 1024,
    repetition_penalty: float = 1.0,
    skip_special_tokens: bool = True,
    default_system: str | None = None,
    enable_thinking: bool = True,
    seed: int | None = None,
    pipeline_parallel_size: int = 1,
    image_max_pixels: int = 768 * 768,
    image_min_pixels: int = 32 * 32,
    video_fps: float = 2.0,
    video_maxlen: int = 128,
    batch_size: int = 1024,
):
    r"""Perform batch generation using vLLM engine, which supports tensor parallelism.

    Usage: python vllm_infer.py --model_name_or_path meta-llama/Llama-2-7b-hf --template llama --dataset alpaca_en_demo
    """
    if pipeline_parallel_size > get_device_count():
        raise ValueError("Pipeline parallel size should be smaller than the number of gpus.")

    model_args, data_args, _, generating_args = get_infer_args(
        dict(
            model_name_or_path=model_name_or_path,
            adapter_name_or_path=adapter_name_or_path,
            dataset=dataset,
            dataset_dir=dataset_dir,
            template=template,
            cutoff_len=cutoff_len,
            max_samples=max_samples,
            preprocessing_num_workers=16,
            default_system=default_system,
            enable_thinking=enable_thinking,
            vllm_config=vllm_config,
            temperature=temperature,
            top_p=top_p,
            top_k=top_k,
            max_new_tokens=max_new_tokens,

View on GitHub (pinned to f28afaf635)

Solutions

pip install mcore-adapter
If you didn't intend the Megatron path, remove the mca/megatron backend selection from the training config so the standard HF path is used

Example fix

# before
# backend selects mca path, package missing -> ImportError

# after
pip install mcore-adapter

Defensive patterns

Strategy: validation

Validate before calling

from llamafactory.extras.packages import is_mcore_adapter_available
if backend == 'mca':
    assert is_mcore_adapter_available(), 'pip install mcore-adapter'

Type guard

def mcore_adapter_ready() -> bool:
    try:
        import mcore_adapter  # noqa: F401
        return True
    except ImportError:
        return False

Try / catch

try:
    run_exp()
except ImportError as e:
    if 'mcore-adapter' in str(e):
        raise SystemExit('pip install mcore-adapter') from e
    raise

Prevention

Install backend extras when the config selects that backend
Document per-backend requirements in your training job templates

When it happens

Trigger: Running the MCA training path (trainers dispatched via train/mca/workflow.py, e.g. Megatron-style SFT/DPO) in an environment without the mcore-adapter wheel; the check fires at import, before any run logic.

Common situations: Selecting the Megatron-Core adapter backend in YAML without installing its extras; fresh environments cloned from a repo that vendors the path but not the dependency.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/246d9a22fc4d2485. Report an issue: GitHub.