hiyouga/LlamaFactory · error · ValueError

`save_dir` already exists, use another one.

Error message

`save_dir` already exists, use another one.

What it means

EvaluationArguments.__post_init__ (src/llamafactory/hparams/evaluation_args.py:60) refuses to overwrite an existing output directory so that previous evaluation results (predictions, metrics) are not silently clobbered. The check runs os.path.exists(save_dir) at argument construction time.

Source

Thrown at src/llamafactory/hparams/evaluation_args.py:60

        default="en",
        metadata={"help": "Language used at evaluation."},
    )
    n_shot: int = field(
        default=5,
        metadata={"help": "Number of exemplars for few-shot learning."},
    )
    save_dir: str | None = field(
        default=None,
        metadata={"help": "Path to save the evaluation results."},
    )
    download_mode: DownloadMode = field(
        default=DownloadMode.REUSE_DATASET_IF_EXISTS,
        metadata={"help": "Download mode used for the evaluation datasets."},
    )

    def __post_init__(self):
        if self.save_dir is not None and os.path.exists(self.save_dir):
            raise ValueError("`save_dir` already exists, use another one.")

View on GitHub (pinned to f28afaf635)

Solutions

1. Delete or move the old directory: rm -rf ./eval_results (after confirming you no longer need it).
2. Point save_dir to a fresh path, e.g. save_dir: eval_results/run2 (timestamped names avoid collisions).
3. In scripts, generate a unique directory per run (e.g. with a timestamp) so reruns never collide.

Example fix

# before (bash)
llamafactory-cli train examples/evaluation/bleu.yaml  # save_dir: saved_results already exists

# after (bash)
rm -rf saved_results
llamafactory-cli train examples/evaluation/bleu.yaml

import os, time

def fresh_save_dir(base: str) -> str:
    d = f"{base}/{time.strftime('%Y%m%d-%H%M%S')}"
    if os.path.exists(d):
        raise FileExistsError(f"refusing to overwrite {d}")
    return d

try:
    eval_args = EvaluationArguments(save_dir="./saved_results", ...)
except ValueError:
    # archive the old results rather than deleting blindly
    os.rename("./saved_results", f"./saved_results.bak-{int(time.time())}")

When it happens

Trigger: Running llamafactory-cli train/eval with an eval config whose save_dir points to a directory that already exists on disk (from a previous run).

Common situations: Re-running an evaluation with the same output path after an earlier run, or a resume/retry after a partially completed evaluation that already created the folder.

Related errors

• `predict_with_generate` cannot be set as True except SFT.
• PPO training does not support evaluation, use the SFT stage
• Checkpoint directory does not exist: {ckpt_dir}
• Please upgrade `transformers` to 4.34.0
• Unable to process key {key}

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/f6502d1889ff21c6. Report an issue: GitHub.