tracel-ai/burn · error

Can load learning rate scheduler checkpoint.

Error message

Can load learning rate scheduler checkpoint.

What it means

In `load_checkpoint`, the learning-rate scheduler record is restored via `restore(epoch)` and unwrapped with expect(). A missing or unreadable scheduler checkpoint for the epoch panics with this message.

Source

Thrown at crates/burn-train/src/learner/base.rs:212

    /// state is migrated to each parameter's device on the next step. The training device is fixed
    /// earlier, when the learner's model is created/forked.
    pub fn load_checkpoint(&self, mut learner: Learner<M>, epoch: usize) -> Learner<M> {
        let record = self
            .model
            .restore(epoch)
            .expect("Can load model checkpoint.");
        learner.load_model(record);

        let record = self
            .optim
            .restore(epoch)
            .expect("Can load optimizer checkpoint.");
        learner.load_optim(record);

        let record = self
            .lr_scheduler
            .restore(epoch)
            .expect("Can load learning rate scheduler checkpoint.");
        learner.load_scheduler(record);

        learner
    }
}

/// Cloneable reference to an early stopping strategy
pub(crate) type EarlyStoppingStrategyRef = Box<dyn CloneEarlyStoppingStrategy>;

#[derive(Clone, Default)]
/// A handle that allows aborting the training/evaluation process early.
pub struct Interrupter {
    state: Arc<AtomicBool>,
    message: Arc<Mutex<Option<String>>>,
}

impl Interrupter {
    /// Create a new instance.

View on GitHub (pinned to d16f7ba2ed)

Solutions

  1. Verify the scheduler checkpoint file for the epoch exists in the scheduler checkpointer's directory.
  2. Use the same scheduler checkpointer directory/prefix and the same LR scheduler configuration as the saving run.
  3. Keep all three checkpoint components saved/deleted together to avoid partial sets.
  4. If the scheduler was intentionally changed, don't restore its checkpoint — construct the learner without restore or provide your own loading path.
  5. Recover the file from a backup snapshot, or restart training from an epoch where the full checkpoint set exists.

Example fix

// before
// scheduler checkpointer never configured on resume
learner::load_checkpoint(learner, epoch); // panics
// after: configure all three checkpointers identically to the saving run
let base = "/checkpoints/run-1";
let checkpointer = LearnerCheckpointer::new(
    FileCheckpointer::new(Recorder, base, "model"),
    FileCheckpointer::new(Recorder, base, "optim"),
    FileCheckpointer::new(Recorder, base, "scheduler"),
);
learner::load_checkpoint(learner, epoch);
Defensive patterns

Strategy: validation

Validate before calling

let sched_path = Path::new(&checkpoint_dir).join(format!("scheduler-{epoch}"));
if !sched_path.exists() {
    eprintln!("scheduler checkpoint missing for epoch {epoch}; configure the scheduler checkpointer or restart LR schedule");
}
assert_eq!(current_scheduler_cfg, saved_scheduler_cfg);

Type guard

fn scheduler_checkpoint_exists(dir: &str, epoch: usize) -> bool {
    Path::new(dir).join(format!("scheduler-{epoch}")).exists()
}

Try / catch

let result = std::panic::catch_unwind(AssertUnwindSafe(|| learner::load_checkpoint(learner, epoch)));
if result.is_err() { eprintln!("scheduler restore failed; restarting LR schedule from scratch"); }

Prevention

When it happens

Trigger: Calling load_checkpoint(learner, epoch) where the LR scheduler checkpoint for that epoch is absent (crash between optimizer and scheduler saves, external deletion) or fails to deserialize (different scheduler type/step settings, burn version mismatch).

Common situations: Resume runs configured without the scheduler checkpointer or with a different directory; switching LR scheduler (e.g. CosineAnnealing → StepLR) between save and resume; cleaned checkpoint dirs that kept only model weights.

Related errors


AI-assisted analysis of tracel-ai/burn@d16f7ba2ed (2026-09-05). Data as JSON: /api/errors/12870c33116b6270. Report an issue: GitHub.