benbjohnson/litestream · error

read dirty page from buffer: %w

Error message

read dirty page from buffer: %w

What it means

In the VFS Read path, if the requested page is dirty, its data is read from the local buffer file at the recorded offset. A ReadAt failure there produces "read dirty page from buffer" and unlocks the mutex before returning. Like error 593, it signals buffer-file/offset inconsistency or local I/O failure.

Source

Thrown at vfs.go:1533

func (f *VFSFile) ReadAt(p []byte, off int64) (n int, err error) {
	f.logger.Debug("reading at", "off", off, "len", len(p))
	pageSize, err := f.pageSizeBytes()
	if err != nil {
		return 0, err
	}

	pgno := uint32(off/int64(pageSize)) + 1
	pageOffset := int(off % int64(pageSize))

	// Check dirty pages first (takes priority over cache and remote)
	f.mu.Lock()
	if f.writeEnabled {
		if bufferOff, ok := f.dirty[pgno]; ok {
			// Read page from buffer file
			data := make([]byte, pageSize)
			if _, err := f.bufferFile.ReadAt(data, bufferOff); err != nil {
				f.mu.Unlock()
				return 0, fmt.Errorf("read dirty page from buffer: %w", err)
			}
			n = copy(p, data[pageOffset:])
			f.mu.Unlock()
			f.logger.Debug("dirty page hit", "page", pgno, "n", n)

			// Update the first page to pretend like we are in journal mode.
			if off == 0 && len(p) >= 28 {
				p[18], p[19] = 0x01, 0x01
				_, _ = rand.Read(p[24:28])
			}

			return n, nil
		}
	}
	f.mu.Unlock()

	// If hydration complete, read from local file
	if f.hydrator != nil && f.hydrator.Complete() {

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Read the wrapped cause; a short read/EOF means stale dirty offsets — invalidate and re-read from remote.
2. Re-open or re-hydrate the VFSFile to rebuild the buffer/dirty map.
3. Ensure all mutations of f.dirty and f.bufferFile happen under f.mu in the same order.
4. Check local disk health and free space for the buffer file.

if bufferOff, ok := f.dirty[pgno]; ok {
    if st, err := f.bufferFile.Stat(); err != nil || bufferOff+int64(pageSize) > st.Size() {
        delete(f.dirty, pgno) // fall back to remote fetch below
    }
}

if _, err := f.bufferFile.ReadAt(data, bufferOff); err != nil {
    f.mu.Unlock()
    if errors.Is(err, io.EOF) {
        return f.readFromRemote(pgno, p) // fallback path
    }
    return 0, err
}

When it happens

Trigger: A page read (ReadAt on the VFS) hitting a f.dirty entry whose buffer offset is beyond the buffer file's current size, or an underlying file read error (disk full, EBADF, tmpfs eviction).

Common situations: Buffer file recreated/compacted without updating f.dirty; local disk errors; running with the buffer on ephemeral storage that was cleared; concurrent code paths mutating f.dirty without holding f.mu.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• read dirty page %d from buffer: %w
• failed to scan directory %s: %w
• copy L0 file: %w
• close L0 file: %w
• page not found: %d

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/1f226edc91d62dce. Report an issue: GitHub.