tracel-ai/burn · error

Can delete model checkpoint.

Error message

Can delete model checkpoint.

What it means

During `Learner::checkpoint`, when a CheckpointingAction::Delete(epoch) is applied, the model checkpointer's `delete(epoch)` result is unwrapped with expect(). If the underlying file recorder fails to delete the model checkpoint file (missing file, IO error, permission problem), the process panics with this message.

Solutions

1. Check that the checkpoint directory passed to the FileCheckpointer still contains the model checkpoint for the target epoch.
2. List the checkpoint dir to confirm the epoch file exists and the current user has write permission on it.
3. Ensure only one training process uses the checkpoint directory at a time.
4. If deletion is best-effort in your setup, wrap checkpointing so a missing epoch is tolerated, or clean up checkpoints yourself instead of via Delete actions.
5. Recreate/restore the checkpoint directory from a backup if files were removed externally.

Example fix

// before
checkpointer.checkpoint(LearnerCheckpointer::new(model_cp, optim_cp, sched_cp));
// with actions [CheckpointingAction::Delete(epoch)] whose file was already removed
// after: verify before scheduling the delete
if std::path::Path::new(&checkpoint_dir).join(format!("model-{epoch}")).exists() {
    actions.push(CheckpointingAction::Delete(epoch));
}
checkpointer.checkpoint(...);

let path = Path::new(&checkpoint_dir).join(format!("model-{epoch}"));
if !path.exists() { eprintln!("model checkpoint for epoch {epoch} missing; skipping delete"); return; }

fn checkpoint_exists(dir: &str, prefix: &str, epoch: usize) -> bool {
    Path::new(dir).join(format!("{prefix}-{epoch}")).exists()
}

// expect() panics are not catchable in Rust without catch_unwind
let result = std::panic::catch_unwind(AssertUnwindSafe(|| checkpointer.checkpoint(actions)));
if result.is_err() { eprintln!("checkpoint delete failed; continuing without cleanup"); }

When it happens

Trigger: Calling checkpointing with a Delete action for an epoch whose model checkpoint file does not exist or cannot be removed by the file recorder (e.g. wrong checkpoint directory, file already deleted, read-only filesystem).

Common situations: Pointing the checkpointer at a different directory than where checkpoints were written; manually cleaning the checkpoint folder while training runs; running two training jobs on the same checkpoint dir; keeping策略 settings (KeepAll / Delete on resume) after the directory contents changed.

Related errors

• Can delete learning rate scheduler checkpoint.
• Can delete optimizer checkpoint.
• Can save learning rate scheduler checkpoint.
• Can save model checkpoint.
• Can save optimizer checkpoint.

AI-assisted analysis of tracel-ai/burn@d16f7ba2ed (2026-09-05). Data as JSON: /api/errors/2163ed482cb07eb3. Report an issue: GitHub.

Appendix: source

Thrown at crates/burn-train/src/learner/base.rs:167

        Self {
            model,
            optim,
            lr_scheduler,
            strategy,
            _phantom: PhantomData,
        }
    }

    /// Create checkpoint for the training process.
    pub fn checkpoint(&mut self, learner: &Learner<M>, epoch: usize, store: &EventStoreClient) {
        let actions = self.strategy.checkpointing(epoch, store);

        for action in actions {
            match action {
                CheckpointingAction::Delete(epoch) => {
                    self.model
                        .delete(epoch)
                        .expect("Can delete model checkpoint.");
                    self.optim
                        .delete(epoch)
                        .expect("Can delete optimizer checkpoint.");
                    self.lr_scheduler
                        .delete(epoch)
                        .expect("Can delete learning rate scheduler checkpoint.");
                }
                CheckpointingAction::Save => {
                    self.model
                        .save(epoch, learner.model.clone().into_record())
                        .expect("Can save model checkpoint.");
                    self.optim
                        .save(epoch, learner.optim.to_record())
                        .expect("Can save optimizer checkpoint.");
                    self.lr_scheduler
                        .save(epoch, learner.lr_scheduler.to_record())
                        .expect("Can save learning rate scheduler checkpoint.");
                }

View on GitHub (pinned to d16f7ba2ed)