{"record":{"id":"3bd61e2cb0d0724b","repo":"benbjohnson/litestream","slug":"read-page-d-from-buffer-for-cache-w","errorCode":null,"errorMessage":"read page %d from buffer for cache: %w","messagePattern":"read page (.+?) from buffer for cache: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"vfs.go","lineNumber":2036,"sourceCode":"\t\t\"size\", info.Size)\n\n\tf.expectedTXID = f.pendingTXID\n\tf.pendingTXID++\n\tf.pos = ltx.Pos{TXID: f.expectedTXID}\n\n\tif f.vfs != nil {\n\t\tf.vfs.writeMu.Lock()\n\t\tif f.expectedTXID > f.vfs.lastSyncedTXID {\n\t\t\tf.vfs.lastSyncedTXID = f.expectedTXID\n\t\t}\n\t\tf.vfs.writeMu.Unlock()\n\t}\n\n\t// Update cache with synced pages (index will be populated naturally when pages are fetched)\n\tfor pgno, bufferOff := range f.dirty {\n\t\tcachedData := make([]byte, f.pageSize)\n\t\tif _, err := f.bufferFile.ReadAt(cachedData, bufferOff); err != nil {\n\t\t\treturn fmt.Errorf(\"read page %d from buffer for cache: %w\", pgno, err)\n\t\t}\n\t\tf.cache.Add(pgno, cachedData)\n\t}\n\n\t// Apply synced pages to hydrated file if hydration is complete\n\t// Must be done before clearing f.dirty since we need the page offsets\n\tif f.hydrator != nil && f.hydrator.Complete() {\n\t\tif err := f.applySyncedPagesToHydratedFile(); err != nil {\n\t\t\tf.logger.Error(\"failed to apply synced pages to hydrated file\", \"error\", err)\n\t\t\t// Don't fail the sync - hydration will catch up on next poll\n\t\t}\n\t}\n\n\t// Clear dirty pages\n\tf.dirty = make(map[uint32]int64)\n\n\t// Clear write buffer after successful sync\n\tif err := f.clearWriteBuffer(); err != nil {","sourceCodeStart":2018,"sourceCodeEnd":2054,"githubUrl":"https://github.com/benbjohnson/litestream/blob/4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3/vfs.go#L2018-L2054","documentation":"After a successful LTX upload, syncToRemoteWithLock re-reads each synced dirty page from the local write-buffer file to populate the in-memory page cache, and a ReadAt on the buffer file failed. The upload already succeeded remotely, but the sync returns an error and f.dirty is not cleared, so pages may be re-uploaded on the next sync.","triggerScenarios":"The buffer file was truncated, cleared, closed, or its offsets invalidated between the LTX creation and the cache-refresh loop — e.g. concurrent clearWriteBuffer, buffer file closed by disable/enable race, or I/O error reading the temp file at the recorded dirty offset.","commonSituations":"tmp cleaner removing/truncating the write-buffer file mid-sync; calling SetWriteEnabled(false)/(true) concurrently with a periodic sync; disk errors on the temp volume; a crash-restore path that reset the buffer while a sync was in flight.","solutions":["Retry the Sync() — the remote LTX already exists, so the re-upload is idempotent at the same TXID or safely advances.","Do not concurrently disable/enable writes or clear the buffer while a sync is in flight; serialize via SetWriteEnabled.","Verify the write-buffer file still exists and has not been truncated (ls -la the bufferPath; check tmp cleaner policies).","Move the write buffer to a stable local volume not subject to tmp reaping (set WriteBufferPath).","If state is suspect, litestream reset for the database and re-enable writes to rebuild buffer/index."],"exampleFix":"// before\ngo file.Sync()             // periodic sync\ngo file.SetWriteEnabled(false) // races: buffer cleared mid-sync\n// after\nfile.mu-driven sync serialized:\nif err := file.SetWriteEnabledWithTimeout(false, time.Minute); err != nil { ... } // waits for tx & syncs once","handlingStrategy":"retry","validationCode":"// Before syncing, confirm the buffer file is intact:\nfi, err := file.bufferFile.Stat()\nif err != nil || fi.Size() < maxDirtyOffset+len(pageSize) { return fmt.Errorf(\"buffer truncated\") }","typeGuard":null,"tryCatchPattern":"if err := file.Sync(); err != nil {\n    if strings.Contains(err.Error(), \"read page\") && strings.Contains(err.Error(), \"for cache\") {\n        // upload already succeeded; a retry is safe (idempotent LTX write)\n        retrySync()\n    }\n}","preventionTips":["Serialize Sync() with SetWriteEnabled calls; never race them.","Place the write buffer on a stable volume exempt from tmp reaping.","Monitor for external truncation of the buffer file.","After a buffer I/O error, verify remote LTX state before resetting (upload may have succeeded)."],"tags":["go","write-buffer","race","cache","sqlite-vfs"],"backgroundTag":"file-read-failed","analyzedSha":"4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3","analyzedAt":"2026-09-06T18:29:25.564Z","contentChangedAt":"2026-09-06T18:29:25.564Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}