{"record":{"id":"091c815ae76136f5","repo":"hashicorp/nomad","slug":"migration-succeeded-but-failed-to-rename-s-to-s","errorCode":null,"errorMessage":"migration succeeded but failed to rename %s to %s: %w","messagePattern":"migration succeeded but failed to rename (.+?) to (.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"helper/raftutil/migrate.go","lineNumber":148,"sourceCode":"\t}\n\n\t// Close both stores before renaming files.\n\tif err := dst.Close(); err != nil {\n\t\tsrc.Close()\n\t\tcleanupWAL(walDir)\n\t\treturn fmt.Errorf(\"failed to close WAL store: %w\", err)\n\t}\n\n\tif err := src.Close(); err != nil {\n\t\tcleanupWAL(walDir)\n\t\treturn fmt.Errorf(\"failed to close BoltDB store: %w\", err)\n\t}\n\n\t// Rename the old BoltDB file to preserve it as a backup with timestamp.\n\ttimestamp := time.Now().Format(\"20060102-150405\")\n\tbackupPath := fmt.Sprintf(\"%s.migrated.%s\", boltPath, timestamp)\n\tif err := os.Rename(boltPath, backupPath); err != nil {\n\t\treturn fmt.Errorf(\"migration succeeded but failed to rename %s to %s: %w\",\n\t\t\tboltPath, backupPath, err)\n\t}\n\n\tsendProgress(progress, fmt.Sprintf(\"migration complete; old BoltDB file preserved at %s\", backupPath))\n\treturn nil\n}\n\nfunc sendProgress(progress chan<- string, msg string) {\n\tif progress != nil {\n\t\tselect {\n\t\tcase progress <- msg:\n\t\tdefault:\n\t\t\t// Drop message if consumer is slow to avoid blocking migration.\n\t\t}\n\t}\n}\n\nfunc drainProgress(sub <-chan string, parent chan<- string, wg *sync.WaitGroup) {","sourceCodeStart":130,"sourceCodeEnd":166,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/helper/raftutil/migrate.go#L130-L166","documentation":"This is the only post-success failure: the data was fully migrated and verified, but os.Rename of raft.db to raft.db.migrated.<timestamp> failed, so the new WAL directory exists alongside the original BoltDB file. Because raft.db is still present at its original path, the server may boot with the old BoltDB backend; the rename must be completed manually before starting on WAL.","triggerScenarios":"os.Rename(boltPath, backupPath) returns an error: a backup file with the same timestamped name already exists, cross-device rename not possible (backup target on a different filesystem via bind mounts), or permission/EPERM/EXDEV/EBUSY issues on the raft directory.","commonSituations":"Re-running migration within the same second after a prior success (same timestamp); raft dir bind-mounted such that rename crosses devices; antivirus/backup software holding raft.db open on Windows; read-only remount between migration and rename.","solutions":["Manually move the file: mv <raftDir>/raft.db <raftDir>/raft.db.migrated.<timestamp>, then start the server (the wal directory is already complete and verified).","If a same-named backup already exists, use a distinct timestamp suffix for the manual rename.","Ensure raft dir and backup path are on the same filesystem; avoid bind-mount layouts that make rename cross-device.","On Windows, close processes (AV scanners, backup agents) holding raft.db open, then rename."],"exampleFix":"// before (restart server right after failed rename)\nsystemctl start nomad\n// after\nmv /var/lib/nomad/raft.db /var/lib/nomad/raft.db.migrated.20260903-120000\nsystemctl start nomad","handlingStrategy":"fallback","validationCode":"backupPath := fmt.Sprintf(\"%s.migrated.%s\", filepath.Join(raftDir, \"raft.db\"), time.Now().Format(\"20060102-150405\"))\nif _, err := os.Stat(backupPath); err == nil {\n    return fmt.Errorf(\"backup %s already exists; waiting avoids same-second collision\", backupPath)\n}\n// also confirm raft.db and backup target are on the same filesystem\ndevSrc, _, _ := statDev(filepath.Join(raftDir, \"raft.db\"))\ndevDst, _, _ := statDev(backupPath)\nif devSrc != devDst {\n    return fmt.Errorf(\"rename would cross devices\")\n}","typeGuard":null,"tryCatchPattern":"err := raftutil.MigrateToWAL(ctx, raftDir, progress)\nif err != nil && strings.Contains(err.Error(), \"migration succeeded but failed to rename\") {\n    // migration IS complete; finish manually, then start the server:\n    // mv <raftDir>/raft.db <raftDir>/raft.db.migrated.manual\n}","preventionTips":["Do not re-run migration within the same second as a previous success (timestamp collision).","Keep raft dir and backup on one filesystem; avoid cross-device bind mounts.","Disable AV/backup scanners holding raft.db open on Windows during migration.","If this error appears, complete the rename manually before starting the server."],"tags":["filesystem","rename","migration","raft"],"backgroundTag":"rename-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}