{"record":{"id":"2faa980bf14f04fb","repo":"hashicorp/nomad","slug":"failed-to-open-wal-store-w","errorCode":null,"errorMessage":"failed to open WAL store: %w","messagePattern":"failed to open WAL store: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"helper/raftutil/migrate.go","lineNumber":98,"sourceCode":"\t\t\tTimeout: 5 * time.Second,\n\t\t},\n\t\tMsgpackUseNewTimeFormat: true,\n\t})\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to open BoltDB store: %w\", err)\n\t}\n\n\t// Create the destination WAL store.\n\tif err := os.MkdirAll(walDir, 0o700); err != nil {\n\t\tsrc.Close()\n\t\treturn fmt.Errorf(\"failed to create WAL directory: %w\", err)\n\t}\n\n\tdst, err := raftwal.Open(walDir)\n\tif err != nil {\n\t\tsrc.Close()\n\t\tcleanupWAL(walDir)\n\t\treturn fmt.Errorf(\"failed to open WAL store: %w\", err)\n\t}\n\n\t// Copy logs.\n\tlogProgress := make(chan string, 64)\n\twg.Add(1)\n\tgo drainProgress(logProgress, progress, &wg)\n\tif err := migrate.CopyLogs(ctx, dst, src, migrateBatchBytes, logProgress); err != nil {\n\t\tdst.Close()\n\t\tsrc.Close()\n\t\tcleanupWAL(walDir)\n\t\treturn fmt.Errorf(\"failed to copy logs: %w\", err)\n\t}\n\n\t// Copy stable store keys.\n\tstableProgress := make(chan string, 64)\n\twg.Add(1)\n\tgo drainProgress(stableProgress, progress, &wg)\n\tif err := migrate.CopyStable(ctx, dst, src, nil, nil, stableProgress); err != nil {","sourceCodeStart":80,"sourceCodeEnd":116,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/helper/raftutil/migrate.go#L80-L116","documentation":"After creating the WAL directory, MigrateToWAL opens the destination with raftwal.Open(walDir). This error wraps any failure from that open, meaning a valid WAL store could not be initialized in the freshly created directory. On failure the source BoltDB store is closed and the partially created WAL directory is removed so a retry starts clean.","triggerScenarios":"raftwal.Open(walDir) returns an error: the WAL backend cannot initialize its segment files (disk full, I/O error), the directory was corrupted between creation and open, an incompatible/corrupt pre-existing WAL layout, or an internal raft-wal initialization error.","commonSituations":"Disk filled up between preflight checks and the open; underlying storage throwing I/O errors; a leftover/corrupt wal directory from a previous crashed run that os.MkdirAll happily accepted; raft-wal version incompatibility with existing segment files.","solutions":["Read the wrapped error (%w) — fix the underlying cause (usually ENOSPC or EIO) and re-run the migration.","Confirm sufficient free disk space (preflight requires bolt size + 512 MiB, but space can be consumed concurrently).","Remove any partially written <raftDir>/wal directory (cleanupWAL may have failed, e.g. on Windows) and retry.","Check storage health (dmesg / smartctl) if the wrapped error is an I/O error."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"usage, err := disk.Usage(raftDir)\nif err == nil && usage.Free < 512*1024*1024 {\n    return fmt.Errorf(\"only %d bytes free; free space before migrating\", usage.Free)\n}","typeGuard":null,"tryCatchPattern":"err := raftutil.MigrateToWAL(ctx, raftDir, progress)\nif err != nil && strings.Contains(err.Error(), \"failed to open WAL store\") {\n    // safe to retry: cleanupWAL already removed the partial directory\n    os.RemoveAll(filepath.Join(raftDir, \"wal\"))\n    err = raftutil.MigrateToWAL(ctx, raftDir, progress)\n}","preventionTips":["Ensure free disk space > raft.db size + 512 MiB before migrating.","Use healthy local block storage; check dmesg for I/O errors first.","Remove stale wal/ directories from failed runs before retrying.","Keep raft-wal library versions consistent with the server build."],"tags":["filesystem","raft-wal","migration","storage"],"backgroundTag":"open-wal-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}