{"record":{"id":"2d9386a1ede1a800","repo":"benbjohnson/litestream","slug":"read-wal-header-after-expected-truncation-w","errorCode":null,"errorMessage":"read wal header after expected truncation: %w","messagePattern":"read wal header after expected truncation: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"db.go","lineNumber":1723,"sourceCode":"\tinfo.offset = dec.Header().WALOffset + dec.Header().WALSize\n\tinfo.salt1 = dec.Header().WALSalt1\n\tinfo.salt2 = dec.Header().WALSalt2\n\tinfo.prevCommit = dec.Header().Commit\n\n\t// If LTX WAL offset is larger than real WAL then the WAL has been truncated.\n\tif fi, err := os.Stat(db.WALPath()); err != nil {\n\t\treturn info, fmt.Errorf(\"open wal file: %w\", err)\n\t} else if info.offset > fi.Size() {\n\t\texec.state.truncatePassiveFailed = false\n\n\t\t// If we previously synced to the exact end of the WAL, this truncation\n\t\t// is expected (normal checkpoint behavior). Reset position and continue\n\t\t// incrementally rather than triggering a full snapshot. See issue #927.\n\t\tif exec.state.syncedToWALEnd {\n\t\t\t// Read new WAL header to get current salt values\n\t\t\thdr, err := readWALHeader(db.WALPath())\n\t\t\tif err != nil {\n\t\t\t\treturn info, fmt.Errorf(\"read wal header after expected truncation: %w\", err)\n\t\t\t}\n\n\t\t\tinfo.offset = WALHeaderSize\n\t\t\tinfo.salt1 = binary.BigEndian.Uint32(hdr[16:])\n\t\t\tinfo.salt2 = binary.BigEndian.Uint32(hdr[20:])\n\t\t\tinfo.snapshotting = false\n\t\t\tinfo.reason = \"\"\n\t\t\tinfo.clearSyncedToWALEnd = true\n\n\t\t\tdb.Logger.Log(ctx, internal.LevelTrace, \"wal truncated after sync to end (expected checkpoint)\",\n\t\t\t\t\"new_salt1\", info.salt1,\n\t\t\t\t\"new_salt2\", info.salt2)\n\n\t\t\treturn info, nil\n\t\t}\n\n\t\tinfo.reason = \"wal truncated by another process\"\n\t\treturn info, nil","sourceCodeStart":1705,"sourceCodeEnd":1741,"githubUrl":"https://github.com/benbjohnson/litestream/blob/4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3/db.go#L1705-L1741","documentation":"After an expected WAL truncation (normal checkpoint behavior, issue #927), verify reads the new WAL header to learn the current salt values. If readWALHeader fails here, the expected-truncation fast path cannot be taken and the sync errors instead of continuing incrementally.","triggerScenarios":"syncedToWALEnd was true, info.offset exceeded the (truncated) WAL size, and readWALHeader(db.WALPath()) fails — WAL removed or unreadable right after a checkpoint truncated it.","commonSituations":"Checkpoint raced with file removal by an external tool; permissions changed mid-run; transient IO errors on network filesystems.","solutions":["Verify the -wal file exists and is readable at db.WALPath()","Check for external scripts/jobs touching the WAL during checkpoints","Fix the wrapped read error (permissions, mount health) and allow the next sync to retry","If local state is desynced, `litestream reset` and resnapshot"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"hdr, err := readWALHeader(dbPath + \"-wal\")\nif err != nil || len(hdr) < 32 { /* WAL unreadable or too short */ }","typeGuard":null,"tryCatchPattern":"if err := db.Sync(ctx); err != nil {\n    if strings.Contains(err.Error(), \"read wal header\") {\n        // transient: retry with backoff; persistent: reset state\n    }\n}","preventionTips":["Don't run WAL-manipulating scripts while litestream monitors the DB","Use a local filesystem, not NFS, for the monitored database","Retry sync with exponential backoff for transient read errors"],"tags":["wal","checkpoint","truncation","header"],"backgroundTag":"file-read-failed","analyzedSha":"4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3","analyzedAt":"2026-09-06T18:29:25.564Z","contentChangedAt":"2026-09-06T18:29:25.564Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}