hiyouga/LlamaFactory · error · ValueError

`resume_from_checkpoint` will be supported in the future ver

Error message

`resume_from_checkpoint` will be supported in the future version.

What it means

The PPO trainer in LlamaFactory implements its own training loop (it does not reuse HuggingFace Trainer's inner loop), and checkpoint resumption for that custom loop is not implemented yet. Calling `ppo_train(resume_from_checkpoint=...)` with a non-None value is explicitly rejected at src/llamafactory/train/ppo/trainer.py:203. This is a deliberate guard so users do not silently get incorrect resume semantics.

Source

Thrown at src/llamafactory/train/ppo/trainer.py:203

                    self.reward_model = self._prepare_deepspeed(self.reward_model)
            else:
                self.reward_model = self.accelerator.prepare_model(self.reward_model, evaluation_mode=True)

        self.add_callback(FixValueHeadModelCallback)

        if processor is not None:
            self.add_callback(SaveProcessorCallback(processor))

        if finetuning_args.use_badam:
            from badam import BAdamCallback, clip_grad_norm_old_version  # type: ignore

            self.accelerator.clip_grad_norm_ = MethodType(clip_grad_norm_old_version, self.accelerator)
            self.add_callback(BAdamCallback)

    def ppo_train(self, resume_from_checkpoint: Optional[str] = None) -> None:
        r"""Implement training loop for the PPO stage, like _inner_training_loop() in Huggingface's Trainer."""
        if resume_from_checkpoint is not None:
            raise ValueError("`resume_from_checkpoint` will be supported in the future version.")

        total_train_batch_size = (
            self.args.per_device_train_batch_size
            * self.args.gradient_accumulation_steps
            * self.finetuning_args.ppo_buffer_size
            * self.args.world_size
        )
        if self.args.max_steps > 0:
            num_examples = total_train_batch_size * self.args.max_steps
            num_train_epochs = sys.maxsize
            max_steps = self.args.max_steps
            steps_in_epoch = self.args.max_steps
        else:
            len_dataloader = len(self.dataloader)
            num_examples = len(self.dataset)
            num_train_epochs = self.args.num_train_epochs
            max_steps = math.ceil(num_train_epochs * len_dataloader)
            steps_in_epoch = len_dataloader

View on GitHub (pinned to f28afaf635)

Solutions

1. Start the PPO run from scratch instead of resuming: remove resume_from_checkpoint from the config/command.
2. If using a wrapper that always passes a checkpoint, patch it to pass None for the ppo stage.
3. Preserve what you can manually: keep the same model/adapter inputs so the base weights are unchanged, and accept that PPO optimizer/step state restarts.
4. Watch LlamaFactory releases; the message states resume support is planned for a future version.

Example fix

# before
trainer.ppo_train(resume_from_checkpoint="output/ppo_run/checkpoint-500")

# after (PPO restarts from current weights, no step-state resume)
trainer.ppo_train(resume_from_checkpoint=None)

def can_resume_ppo(trainer, resume_from_checkpoint):
    return resume_from_checkpoint is None

try:
    trainer.ppo_train(resume_from_checkpoint=None)
except ValueError as e:
    if "resume_from_checkpoint" in str(e):
        trainer.ppo_train(resume_from_checkpoint=None)  # restart clean
    else:
        raise

When it happens

Trigger: Running `llamafactory-cli train` with a PPO config and either passing `resume_from_checkpoint` in the training args or invoking `trainer.ppo_train(resume_from_checkpoint=<path>)` directly (e.g. from a custom script or via TrainingArguments.auto_find_batch_size-style flows that retry with a checkpoint path).

Common situations: A user's RLHF/PPO run is interrupted and they try to resume it the way they would resume an SFT run (`--resume_from_checkpoint checkpoint-xxx`). Any framework or wrapper that always passes a checkpoint path to train() will trip this on the PPO stage.

Related errors

• Output directory already exists and is not empty. Please set
• Unexpected missing keys when loading checkpoint model weight
• Checkpoint directory does not exist: {ckpt_dir}
• Invalid role
• `reward_model` is necessary for PPO training.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/794234c82b67b860. Report an issue: GitHub.