benbjohnson/litestream · error

read page %d from buffer for cache: %w

Error message

read page %d from buffer for cache: %w

What it means

After a successful LTX upload, syncToRemoteWithLock re-reads each synced dirty page from the local write-buffer file to populate the in-memory page cache, and a ReadAt on the buffer file failed. The upload already succeeded remotely, but the sync returns an error and f.dirty is not cleared, so pages may be re-uploaded on the next sync.

Source

Thrown at vfs.go:2036

		"size", info.Size)

	f.expectedTXID = f.pendingTXID
	f.pendingTXID++
	f.pos = ltx.Pos{TXID: f.expectedTXID}

	if f.vfs != nil {
		f.vfs.writeMu.Lock()
		if f.expectedTXID > f.vfs.lastSyncedTXID {
			f.vfs.lastSyncedTXID = f.expectedTXID
		}
		f.vfs.writeMu.Unlock()
	}

	// Update cache with synced pages (index will be populated naturally when pages are fetched)
	for pgno, bufferOff := range f.dirty {
		cachedData := make([]byte, f.pageSize)
		if _, err := f.bufferFile.ReadAt(cachedData, bufferOff); err != nil {
			return fmt.Errorf("read page %d from buffer for cache: %w", pgno, err)
		}
		f.cache.Add(pgno, cachedData)
	}

	// Apply synced pages to hydrated file if hydration is complete
	// Must be done before clearing f.dirty since we need the page offsets
	if f.hydrator != nil && f.hydrator.Complete() {
		if err := f.applySyncedPagesToHydratedFile(); err != nil {
			f.logger.Error("failed to apply synced pages to hydrated file", "error", err)
			// Don't fail the sync - hydration will catch up on next poll
		}
	}

	// Clear dirty pages
	f.dirty = make(map[uint32]int64)

	// Clear write buffer after successful sync
	if err := f.clearWriteBuffer(); err != nil {

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Retry the Sync() — the remote LTX already exists, so the re-upload is idempotent at the same TXID or safely advances.
2. Do not concurrently disable/enable writes or clear the buffer while a sync is in flight; serialize via SetWriteEnabled.
3. Verify the write-buffer file still exists and has not been truncated (ls -la the bufferPath; check tmp cleaner policies).
4. Move the write buffer to a stable local volume not subject to tmp reaping (set WriteBufferPath).
5. If state is suspect, litestream reset for the database and re-enable writes to rebuild buffer/index.

Example fix

// before
go file.Sync()             // periodic sync
go file.SetWriteEnabled(false) // races: buffer cleared mid-sync
// after
file.mu-driven sync serialized:
if err := file.SetWriteEnabledWithTimeout(false, time.Minute); err != nil { ... } // waits for tx & syncs once

// Before syncing, confirm the buffer file is intact:
fi, err := file.bufferFile.Stat()
if err != nil || fi.Size() < maxDirtyOffset+len(pageSize) { return fmt.Errorf("buffer truncated") }

if err := file.Sync(); err != nil {
    if strings.Contains(err.Error(), "read page") && strings.Contains(err.Error(), "for cache") {
        // upload already succeeded; a retry is safe (idempotent LTX write)
        retrySync()
    }
}

When it happens

Trigger: The buffer file was truncated, cleared, closed, or its offsets invalidated between the LTX creation and the cache-refresh loop — e.g. concurrent clearWriteBuffer, buffer file closed by disable/enable race, or I/O error reading the temp file at the recorded dirty offset.

Common situations: tmp cleaner removing/truncating the write-buffer file mid-sync; calling SetWriteEnabled(false)/(true) concurrently with a periodic sync; disk errors on the temp volume; a crash-restore path that reset the buffer while a sync was in flight.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• write to buffer: %w
• init write buffer: %w
• page size mismatch: got %d, expected %d
• context cancelled while waiting for transaction: %w
• timeout waiting for transaction to complete (waited %v)

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/3bd61e2cb0d0724b. Report an issue: GitHub.