hiyouga/LlamaFactory · error · ValueError

Output directory already exists and is not empty. Please set

Error message

Output directory already exists and is not empty. Please set `overwrite_output_dir`.

What it means

Raised in get_train_args when do_train is true, the output_dir already exists, resume_from_checkpoint is unset, overwrite_output_dir is false, and no resumable checkpoint (no file named in CHECKPOINT_NAMES) can be found in it. LlamaFactory refuses to silently mix a new run's artifacts with leftovers from a previous run in the same directory.

Source

Thrown at src/llamafactory/hparams/parser.py:623

        can_resume_from_checkpoint = False
        if training_args.resume_from_checkpoint is not None:
            logger.warning_rank0("Cannot resume from checkpoint in current stage.")
            training_args.resume_from_checkpoint = None
    else:
        can_resume_from_checkpoint = True

    if (
        training_args.resume_from_checkpoint is None
        and training_args.do_train
        and os.path.isdir(training_args.output_dir)
        and not getattr(training_args, "overwrite_output_dir", False)  # for mca training args and transformers >= 5.0
        and can_resume_from_checkpoint
    ):
        last_checkpoint = get_last_checkpoint(training_args.output_dir)
        if last_checkpoint is None and any(
            os.path.isfile(os.path.join(training_args.output_dir, name)) for name in CHECKPOINT_NAMES
        ):
            raise ValueError("Output directory already exists and is not empty. Please set `overwrite_output_dir`.")

        if last_checkpoint is not None:
            training_args.resume_from_checkpoint = last_checkpoint
            logger.info_rank0(f"Resuming training from {training_args.resume_from_checkpoint}.")
            logger.info_rank0("Change `output_dir` or use `overwrite_output_dir` to avoid.")

    if (
        finetuning_args.stage in ["rm", "ppo"]
        and finetuning_args.finetuning_type == "lora"
        and training_args.resume_from_checkpoint is not None
    ):
        logger.warning_rank0(
            f"Add {training_args.resume_from_checkpoint} to `adapter_name_or_path` to resume training from checkpoint."
        )

    # Post-process model arguments
    if training_args.bf16 or finetuning_args.pure_bf16:
        model_args.compute_dtype = torch.bfloat16

View on GitHub (pinned to f28afaf635)

Solutions

1. Set overwrite_output_dir: true in the training section of the YAML config.
2. Point output_dir to a fresh, empty directory for the new run.
3. Delete or move the stale files in the existing output_dir (only if you no longer need them).
4. If you meant to resume, ensure the directory actually contains a valid checkpoint (e.g. checkpoint-500 with trainer_state.json); otherwise the run cannot auto-resume.

Example fix

# before
output_dir: saves/llama3-lora
# -> ValueError if saves/llama3-lora has leftover files

# after
output_dir: saves/llama3-lora
overwrite_output_dir: true

import os
from transformers.trainer_utils import get_last_checkpoint

ckpt = get_last_checkpoint(output_dir) if os.path.isdir(output_dir) else None
if ckpt is None and os.path.isdir(output_dir) and os.listdir(output_dir):
    # decide explicitly: wipe or fail
    assert overwrite_output_dir, f"{output_dir} non-empty and not resumable; set overwrite_output_dir or clean it"

When it happens

Trigger: Running llamafactory-cli train twice with the same output_dir after the first run crashed before writing a checkpoint, or pointing output_dir at a non-empty directory that contains stray files (e.g. a tokenizer export or a log) but no trainer checkpoint.

Common situations: Re-running a failed experiment into the same folder; resuming after an OOM kill that happened before the first checkpoint was saved; reusing a directory that was partially cleaned with rm but still holds files.

Related errors

• `resume_from_checkpoint` will be supported in the future ver
• Unexpected missing keys when loading checkpoint model weight
• Checkpoint directory does not exist: {ckpt_dir}
• KTransformers is incompatible with DeepSpeed ZeRO-3.
• {self.__class__.__name__} does not support gradient checkpoi

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/444473a6cb4a065d. Report an issue: GitHub.