{"record":{"id":"47603729e9a6fdab","repo":"hashicorp/nomad","slug":"failed-to-copy-logs-w","errorCode":null,"errorMessage":"failed to copy logs: %w","messagePattern":"failed to copy logs: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"helper/raftutil/migrate.go","lineNumber":109,"sourceCode":"\t\treturn fmt.Errorf(\"failed to create WAL directory: %w\", err)\n\t}\n\n\tdst, err := raftwal.Open(walDir)\n\tif err != nil {\n\t\tsrc.Close()\n\t\tcleanupWAL(walDir)\n\t\treturn fmt.Errorf(\"failed to open WAL store: %w\", err)\n\t}\n\n\t// Copy logs.\n\tlogProgress := make(chan string, 64)\n\twg.Add(1)\n\tgo drainProgress(logProgress, progress, &wg)\n\tif err := migrate.CopyLogs(ctx, dst, src, migrateBatchBytes, logProgress); err != nil {\n\t\tdst.Close()\n\t\tsrc.Close()\n\t\tcleanupWAL(walDir)\n\t\treturn fmt.Errorf(\"failed to copy logs: %w\", err)\n\t}\n\n\t// Copy stable store keys.\n\tstableProgress := make(chan string, 64)\n\twg.Add(1)\n\tgo drainProgress(stableProgress, progress, &wg)\n\tif err := migrate.CopyStable(ctx, dst, src, nil, nil, stableProgress); err != nil {\n\t\tdst.Close()\n\t\tsrc.Close()\n\t\tcleanupWAL(walDir)\n\t\treturn fmt.Errorf(\"failed to copy stable store: %w\", err)\n\t}\n\n\t// Verify data integrity before finalizing.\n\tsendProgress(progress, \"verifying migrated data\")\n\tif err := verifyMigration(src, dst); err != nil {\n\t\tdst.Close()\n\t\tsrc.Close()","sourceCodeStart":91,"sourceCodeEnd":127,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/helper/raftutil/migrate.go#L91-L127","documentation":"migrate.CopyLogs copies all raft log entries from the BoltDB source into the new WAL store in ~64 MiB batches, respecting ctx cancellation. This error wraps any failure during that copy — a read failure from BoltDB, a write/flush failure into the WAL, or a context cancellation. On failure both stores are closed and the WAL directory is removed, leaving raft.db intact for retry.","triggerScenarios":"migrate.CopyLogs(ctx, dst, src, migrateBatchBytes, logProgress) returns an error: corrupted BoltDB log entries, disk full while writing WAL segments, I/O errors on either store, or ctx cancelled/expired mid-copy.","commonSituations":"Disk runs out during the copy of a large raft.db; operator cancels the context or the migration command times out on a very large log; BoltDB file corrupted by a previous crash; flaky network storage drops I/O mid-copy.","solutions":["Read the wrapped error — for context cancellation, re-run with a longer-lived context and no artificial deadline.","Free disk space: the copy temporarily needs space for both formats (bolt size + 512 MiB buffer at minimum).","Verify BoltDB integrity (bolt check on raft.db) if the wrapped error indicates a read/corruption failure.","Re-run MigrateToWAL — it is safe to retry because the failed WAL directory is cleaned up automatically."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"if boltFile, err := os.Open(filepath.Join(raftDir, \"raft.db\")); err == nil {\n    boltFile.Close()\n}\n// also ensure free space for the WAL copy:\n// free space >= raft.db size + 512 MiB","typeGuard":null,"tryCatchPattern":"ctx, cancel := context.WithTimeout(context.Background(), maxMigrationDuration)\ndefer cancel()\nerr := raftutil.MigrateToWAL(ctx, raftDir, progress)\nif err != nil {\n    if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {\n        // retry with a longer window; source raft.db is untouched\n    }\n}","preventionTips":["Stop the Nomad server before migrating so the log is quiescent.","Do not place short deadlines on the migration context.","Monitor free disk space during migration of large raft.db files.","Run bolt check on raft.db if corruption errors appear."],"tags":["migration","raft","io","context-canceled"],"backgroundTag":"log-copy-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}