hiyouga/LlamaFactory · error · ValueError

Unknown task: {finetuning_args.stage}.

Error message

Unknown task: {finetuning_args.stage}.

What it means

At the end of `_training_function` (src/llamafactory/train/tuner.py:151) the stage string is dispatched over exactly six values: pt, sft, rm, ppo, dpo, kto. Anything else raises ValueError naming the offending stage. This almost always means the YAML `stage:` key was misspelled or the args dict was built incorrectly.

Source

Thrown at src/llamafactory/train/tuner.py:151

        elif finetuning_args.stage == "dpo":
            from .mca import run_dpo as run_dpo_mca

            run_dpo_mca(model_args, data_args, training_args, finetuning_args, callbacks)

    elif finetuning_args.stage == "pt":
        run_pt(model_args, data_args, training_args, finetuning_args, callbacks)
    elif finetuning_args.stage == "sft":
        run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
    elif finetuning_args.stage == "rm":
        run_rm(model_args, data_args, training_args, finetuning_args, callbacks)
    elif finetuning_args.stage == "ppo":
        run_ppo(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
    elif finetuning_args.stage == "dpo":
        run_dpo(model_args, data_args, training_args, finetuning_args, callbacks)
    elif finetuning_args.stage == "kto":
        run_kto(model_args, data_args, training_args, finetuning_args, callbacks)
    else:
        raise ValueError(f"Unknown task: {finetuning_args.stage}.")

    if is_ray_available() and ray.is_initialized():
        return  # if ray is initialized it will destroy the process group on return

    try:
        if dist.is_initialized():
            dist.destroy_process_group()
    except Exception as e:
        logger.warning(f"Failed to destroy process group: {e}.")


def run_exp(args: Optional[dict[str, Any]] = None, callbacks: Optional[list["TrainerCallback"]] = None) -> None:
    args = read_args(args)
    if "-h" in args or "--help" in args:
        get_train_args(args)

    ray_args = get_ray_args(args)
    callbacks = callbacks or []

View on GitHub (pinned to f28afaf635)

Solutions

1. Set `stage:` to one of: pt, sft, rm, ppo, dpo, kto (lowercase) in the YAML.
2. If generating configs programmatically, validate the stage against that enum before invoking run_exp.
3. Check for YAML indentation errors that make `stage` parse as part of another key.

Example fix

# before (yaml)
stage: SFT

# after
stage: sft

VALID_STAGES = {"pt", "sft", "rm", "ppo", "dpo", "kto"}
assert stage in VALID_STAGES, f"stage must be one of {VALID_STAGES}"

def is_valid_stage(stage: str) -> bool:
    return stage in {"pt", "sft", "rm", "ppo", "dpo", "kto"}

When it happens

Trigger: A training YAML with `stage: SFT` (uppercase), `stage: pretrain`, `stage: rm_training`, or a typo like `stage: sfy`; or calling run_exp with a hand-built dict lacking a valid stage.

Common situations: Copy-paste edits of example configs; users renaming stages from other frameworks (e.g. 'instruction_tuning'); programmatic config generation.

Related errors

• Template {data_args.template} does not exist.
• Tool utils `{name}` not found.
• `predict_with_generate` cannot be set as True except SFT.
• `neat_packing` cannot be set as True except SFT.
• `train_on_prompt` or `mask_history` cannot be set as True ex

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/16cfbd1657d73835. Report an issue: GitHub.