benbjohnson/litestream · error

read page %d from buffer: %w

Error message

read page %d from buffer: %w

What it means

When encoding an LTX transaction, each dirty page's data is read back from the write-buffer file at the offset recorded when the page was written. This error wraps bufferFile.ReadAt failing for a specific page number, meaning the write buffer is unreadable, truncated, or the recorded offset is beyond the file's end.

Source

Thrown at vfs.go:2158

			MaxTXID:   pendingTXID,
			Timestamp: time.Now().UnixMilli(),
		}); err != nil {
			err = fmt.Errorf("encode header: %w", err)
			return
		}

		// Encode each dirty page
		lockPgno := ltx.LockPgno(pageSize)
		for _, pgno := range pgnos {
			if pgno == lockPgno {
				continue // Skip lock page
			}

			// Read page data from buffer file
			bufferOff := dirtyOffsets[pgno]
			data := make([]byte, pageSize)
			if _, err = bufferFile.ReadAt(data, bufferOff); err != nil {
				err = fmt.Errorf("read page %d from buffer: %w", pgno, err)
				return
			}

			if err = enc.EncodePage(ltx.PageHeader{Pgno: pgno}, data); err != nil {
				err = fmt.Errorf("encode page %d: %w", pgno, err)
				return
			}
		}

		// Close encoder (writes trailer and page index)
		if err = enc.Close(); err != nil {
			err = fmt.Errorf("close encoder: %w", err)
			return
		}
	}()

	return pr
}

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Verify the buffer file still exists and its size matches expectations; restore it by restarting the process (buffer is rebuilt on open).
2. Ensure no external process truncates or rotates the buffer file path (f.bufferPath).
3. Check the wrapped error: EOF/ErrUnexpectedEOF implies dirty-offset/bookkeeping drift — restart the database handle; EIO implies failing disk.
4. Keep the database and its VFS state confined to a single process; do not share the buffer path across instances.

Example fix

// before: logrotate matching *.buffer truncates live buffer files
// /etc/logrotate.d/app: /var/tmp/*.buffer { copytruncate }
// after: exclude litestream buffer files from rotation
// /var/tmp/*.log { copytruncate }

// confirm buffer file exists and is not truncated externally
if fi, err := os.Stat(bufferPath); err != nil || fi.Size() == 0 { return errors.New("write buffer missing/empty; restart required") }

if err := db.Sync(ctx); err != nil {
    if strings.Contains(err.Error(), "read page") {
        logger.Error("write buffer unreadable; reopen DB to rebuild buffer", "err", err)
    }
    return err
}

When it happens

Trigger: bufferFile.ReadAt(data, bufferOff) returns an error during sync: buffer file deleted/truncated underneath the VFS, offset from f.dirty exceeding buffer length (buffer reset without clearing dirty map), or underlying I/O error.

Common situations: External cleanup jobs removing the buffer file; crash between buffer truncation and dirty-map reset; memory/disk corruption on tmp volumes; mixing processes against the same buffer file path.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• clear write buffer: %w
• write snapshot ltx: %w
• create temp dir for hydration: %w
• cannot delete vfs file
• invalid temp file name: %q

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/3635077ba23a5f97. Report an issue: GitHub.