benbjohnson/litestream · error

read page from buffer

Error message

read page %d from buffer: %w

What it means

When encoding an LTX transaction, each dirty page's data is read back from the write-buffer file at the offset recorded when the page was written. This error wraps bufferFile.ReadAt failing for a specific page number, meaning the write buffer is unreadable, truncated, or the recorded offset is beyond the file's end.

Solutions

1. Verify the buffer file still exists and its size matches expectations; restore it by restarting the process (buffer is rebuilt on open).
2. Ensure no external process truncates or rotates the buffer file path (f.bufferPath).
3. Check the wrapped error: EOF/ErrUnexpectedEOF implies dirty-offset/bookkeeping drift — restart the database handle; EIO implies failing disk.
4. Keep the database and its VFS state confined to a single process; do not share the buffer path across instances.

Example fix

// before: logrotate matching *.buffer truncates live buffer files
// /etc/logrotate.d/app: /var/tmp/*.buffer { copytruncate }
// after: exclude litestream buffer files from rotation
// /var/tmp/*.log { copytruncate }

// confirm buffer file exists and is not truncated externally
if fi, err := os.Stat(bufferPath); err != nil || fi.Size() == 0 { return errors.New("write buffer missing/empty; restart required") }

if err := db.Sync(ctx); err != nil {
    if strings.Contains(err.Error(), "read page") {
        logger.Error("write buffer unreadable; reopen DB to rebuild buffer", "err", err)
    }
    return err
}

When it happens

Trigger: bufferFile.ReadAt(data, bufferOff) returns an error during sync: buffer file deleted/truncated underneath the VFS, offset from f.dirty exceeding buffer length (buffer reset without clearing dirty map), or underlying I/O error.

Common situations: External cleanup jobs removing the buffer file; crash between buffer truncation and dirty-map reset; memory/disk corruption on tmp volumes; mixing processes against the same buffer file path.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• clear write buffer
• cannot delete vfs file
• could not parse time
• create buffer directory
• create temp dir for hydration

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/3635077ba23a5f97. Report an issue: GitHub.

Appendix: source

Thrown at vfs.go:2158

			MaxTXID:   pendingTXID,
			Timestamp: time.Now().UnixMilli(),
		}); err != nil {
			err = fmt.Errorf("encode header: %w", err)
			return
		}

		// Encode each dirty page
		lockPgno := ltx.LockPgno(pageSize)
		for _, pgno := range pgnos {
			if pgno == lockPgno {
				continue // Skip lock page
			}

			// Read page data from buffer file
			bufferOff := dirtyOffsets[pgno]
			data := make([]byte, pageSize)
			if _, err = bufferFile.ReadAt(data, bufferOff); err != nil {
				err = fmt.Errorf("read page %d from buffer: %w", pgno, err)
				return
			}

			if err = enc.EncodePage(ltx.PageHeader{Pgno: pgno}, data); err != nil {
				err = fmt.Errorf("encode page %d: %w", pgno, err)
				return
			}
		}

		// Close encoder (writes trailer and page index)
		if err = enc.Close(); err != nil {
			err = fmt.Errorf("close encoder: %w", err)
			return
		}
	}()

	return pr
}

View on GitHub (pinned to 4ed7a308f6)