{"record":{"id":"b95888c2481d62bf","repo":"tracel-ai/burn","slug":"can-load-optimizer-checkpoint","errorCode":null,"errorMessage":"Can load optimizer checkpoint.","messagePattern":"Can load optimizer checkpoint\\.","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/burn-train/src/learner/base.rs","lineNumber":206,"sourceCode":"    }\n\n    /// Load a training checkpoint.\n    ///\n    /// No device is taken: checkpoints are device-free burnpack records (file-backed bytes). On\n    /// load, the model keeps the device of the learner's existing parameters, and the optimizer\n    /// state is migrated to each parameter's device on the next step. The training device is fixed\n    /// earlier, when the learner's model is created/forked.\n    pub fn load_checkpoint(&self, mut learner: Learner<M>, epoch: usize) -> Learner<M> {\n        let record = self\n            .model\n            .restore(epoch)\n            .expect(\"Can load model checkpoint.\");\n        learner.load_model(record);\n\n        let record = self\n            .optim\n            .restore(epoch)\n            .expect(\"Can load optimizer checkpoint.\");\n        learner.load_optim(record);\n\n        let record = self\n            .lr_scheduler\n            .restore(epoch)\n            .expect(\"Can load learning rate scheduler checkpoint.\");\n        learner.load_scheduler(record);\n\n        learner\n    }\n}\n\n/// Cloneable reference to an early stopping strategy\npub(crate) type EarlyStoppingStrategyRef = Box<dyn CloneEarlyStoppingStrategy>;\n\n#[derive(Clone, Default)]\n/// A handle that allows aborting the training/evaluation process early.\npub struct Interrupter {","sourceCodeStart":188,"sourceCodeEnd":224,"githubUrl":"https://github.com/tracel-ai/burn/blob/d16f7ba2ed0d41408189384044cc886fb4c8f957/crates/burn-train/src/learner/base.rs#L188-L224","documentation":"In `load_checkpoint`, the optimizer record is restored via the optimizer checkpointer's `restore(epoch)` and unwrapped with expect(). A missing or unreadable optimizer checkpoint for the epoch panics with this message.","triggerScenarios":"Calling load_checkpoint(learner, epoch) where the model checkpoint exists but the optimizer checkpoint for that epoch does not (partial save from a crash, external cleanup removed optim files) or fails to deserialize (optimizer changed between runs, e.g. switched Adam to SGD, or version mismatch).","commonSituations":"Model and optimizer checkpointers pointed at different directories on resume; disk cleanup deleted the large optimizer files but kept model weights; changing the optimizer type while resuming.","solutions":["Verify the optimizer checkpoint file for the epoch exists in the optimizer checkpointer's directory.","Point the optimizer checkpointer at the same directory/prefix used by the saving run.","If the optimizer type or burn version changed, re-match it or start training fresh (or accept degraded resume from weights only via your own load path).","Keep model/optimizer/scheduler saves atomic so a crash can't leave only model files.","Restore the missing optimizer file from backup if you have periodic storage snapshots."],"exampleFix":"// before\nlet optim_cp = FileCheckpointer::new(Recorder, \"/other/dir\", \"optim\"); // wrong dir on resume\n// after\nlet base = \"/checkpoints/run-1\"; // same as the saving run\nlet optim_cp = FileCheckpointer::new(Recorder, base, \"optim\");\nlearner::load_checkpoint(learner, epoch);","handlingStrategy":"validation","validationCode":"let optim_path = Path::new(&checkpoint_dir).join(format!(\"optim-{epoch}\"));\nassert!(optim_path.exists(), \"optimizer checkpoint missing for epoch {epoch}\");\n// also confirm the optimizer type matches the one used at save time\nassert_eq!(current_optimizer_name, saved_optimizer_name);","typeGuard":"fn optimizer_checkpoint_exists(dir: &str, epoch: usize) -> bool {\n    Path::new(dir).join(format!(\"optim-{epoch}\")).exists()\n}","tryCatchPattern":"let result = std::panic::catch_unwind(AssertUnwindSafe(|| learner::load_checkpoint(learner, epoch)));\nif result.is_err() { eprintln!(\"optimizer restore failed; resuming with fresh optimizer state\"); }","preventionTips":["Keep optimizer checkpoint files in the same directory as model weights; cleanup scripts often remove only the large optim files.","Don't change optimizer type between save and resume; if you must, restart training or skip optimizer restore explicitly.","Check all three components exist before resuming.","Snapshot checkpoint dirs so partial/corrupt sets can be recovered."],"tags":["rust","panic","checkpoint","optimizer-state","deserialization","resume-training"],"backgroundTag":"checkpoint-not-found","analyzedSha":"d16f7ba2ed0d41408189384044cc886fb4c8f957","analyzedAt":"2026-09-05T13:19:14.260Z","contentChangedAt":"2026-09-05T13:19:14.260Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}