{"record":{"id":"0d330cc51ff3f44d","repo":"benbjohnson/litestream","slug":"read-dirty-page-d-from-buffer-w","errorCode":null,"errorMessage":"read dirty page %d from buffer: %w","messagePattern":"read dirty page (.+?) from buffer: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"vfs.go","lineNumber":1439,"sourceCode":"\t\t\treturn\n\t\t}\n\t}\n\n\tf.hydrator.SetComplete()\n\n\t// Clear cache since we'll now read from hydration file\n\tf.cache.Purge()\n\n\tf.logger.Debug(\"hydration complete\", \"path\", f.hydrationPath, \"txid\", f.hydrator.TXID().String())\n}\n\n// applySyncedPagesToHydratedFile writes synced dirty pages to the hydrated file.\n// Must be called with f.mu held.\nfunc (f *VFSFile) applySyncedPagesToHydratedFile() error {\n\tfor pgno, bufferOff := range f.dirty {\n\t\tdata := make([]byte, f.pageSize)\n\t\tif _, err := f.bufferFile.ReadAt(data, bufferOff); err != nil {\n\t\t\treturn fmt.Errorf(\"read dirty page %d from buffer: %w\", pgno, err)\n\t\t}\n\n\t\tif err := f.hydrator.WritePage(pgno, data); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tf.hydrator.SetTXID(f.expectedTXID)\n\treturn nil\n}\n\nfunc (f *VFSFile) Close() error {\n\tf.logger.Debug(\"closing file\")\n\n\t// Stop sync loop and ticker if running (need mutex for syncStop)\n\tf.mu.Lock()\n\tif f.syncStop != nil {\n\t\tclose(f.syncStop)","sourceCodeStart":1421,"sourceCodeEnd":1457,"githubUrl":"https://github.com/benbjohnson/litestream/blob/4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3/vfs.go#L1421-L1457","documentation":"applySyncedPagesToHydratedFile copies dirty pages from the local buffer file into the hydrated database file. Reading a dirty page at its recorded buffer offset failed with a wrapped \"read dirty page %d from buffer\" error. This indicates the local buffer file is shorter than expected or its offsets are inconsistent with the dirty map.","triggerScenarios":"After sync, iterating f.dirty and calling bufferFile.ReadAt when the buffer file was truncated/recreated, the offset is stale (buffer rebuilt without clearing f.dirty), or an OS-level read error occurs (disk full, I/O error).","commonSituations":"Crash or restart leaving f.dirty entries pointing past the end of a recreated buffer file; disk-full conditions on the local scratch volume; tmpfs cleared under the process; bugs in buffer compaction not invalidating dirty offsets.","solutions":["Check the wrapped error: io.EOF/short read means stale offsets — clear f.dirty and re-hydrate.","Verify the local buffer/scratch directory has free space and is writable.","Recreate the VFSFile (re-hydrate from replica) to rebuild buffer and dirty map consistently.","Report/persist buffer file and dirty-map lifecycle if offsets can survive buffer rebuilds."],"exampleFix":"// before: buffer rebuilt but dirty map not cleared\nf.bufferFile = newBuffer()\n// stale offsets in f.dirty now point past EOF\n\n// after: invalidate dirty entries when the buffer is recreated\nf.bufferFile = newBuffer()\nf.dirty = make(map[uint32]int64)","handlingStrategy":"validation","validationCode":"stat, err := f.bufferFile.Stat()\nif err != nil { return err }\nfor pgno, off := range f.dirty {\n    if off < 0 || off+int64(f.pageSize) > stat.Size() {\n        delete(f.dirty, pgno) // stale offset; re-read from remote instead\n    }\n}","typeGuard":null,"tryCatchPattern":"if err := f.applySyncedPagesToHydratedFile(); err != nil {\n    if errors.Is(err, io.EOF) || errors.Is(err, io.ErrUnexpectedEOF) {\n        f.dirty = make(map[uint32]int64) // rebuild buffer/dirty state\n        err = f.hydrate(ctx)\n    }\n}","preventionTips":["Always clear f.dirty when recreating or compacting the buffer file.","Monitor free space on the volume holding the buffer file.","Avoid ephemeral (tmpfs/container-layer) storage for long-lived buffer files."],"tags":["io","buffer-file","hydration","litestream"],"backgroundTag":"file-read-failed","analyzedSha":"4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3","analyzedAt":"2026-09-06T18:29:25.564Z","contentChangedAt":"2026-09-06T18:29:25.564Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}