benbjohnson/litestream · error

read hydrated file: %w

Error message

read hydrated file: %w

What it means

Hydrator.ReadAt wraps any non-EOF error from reading the local hydrated database file as "read hydrated file". Litestream's VFS serves SQLite reads from a locally hydrated copy of the database; if the underlying file read fails (bad fd, I/O error, out-of-range read on a closed/damaged file), the failure is wrapped with this message and returned to the SQLite layer.

Source

Thrown at vfs.go:884

		}

		off := int64(phdr.Pgno-1) * int64(h.pageSize)
		if _, err := h.file.WriteAt(data, off); err != nil {
			return fmt.Errorf("write page %d: %w", phdr.Pgno, err)
		}
	}

	return nil
}

// ReadAt reads data from the hydrated local file.
func (h *Hydrator) ReadAt(p []byte, off int64) (int, error) {
	h.mu.Lock()
	n, err := h.file.ReadAt(p, off)
	h.mu.Unlock()

	if err != nil && err != io.EOF {
		return n, fmt.Errorf("read hydrated file: %w", err)
	}

	// Update the first page to pretend like we are in journal mode
	if off == 0 && len(p) >= 28 {
		p[18], p[19] = 0x01, 0x01
		_, _ = rand.Read(p[24:28])
	}

	return n, nil
}

// ApplyUpdates fetches updated pages and writes them to the hydration file.
func (h *Hydrator) ApplyUpdates(ctx context.Context, updates map[uint32]ltx.PageIndexElem) error {
	h.mu.Lock()
	defer h.mu.Unlock()

	for pgno, elem := range updates {
		_, data, err := FetchPage(ctx, h.client, elem.Level, elem.MinTXID, elem.MaxTXID, elem.Offset, elem.Size)

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Check the wrapped cause (%w) for os.PathError details — verify the hydration file path exists and is readable by the litestream process
2. Verify disk health and free space on the volume holding the hydration file (dmesg / SMART, df)
3. Check ulimit -n / file descriptor limits if errors appear under load
4. Use litestream reset for the database to clear corrupted local hydration state and re-hydrate from the replica

Example fix

// before
n, err := h.file.ReadAt(p, off)
// after
if n, err = h.file.ReadAt(p, off); err != nil && err != io.EOF {
	return n, fmt.Errorf("read hydrated file (off=%d, len=%d): %w", off, len(p), err)
}

// Go: before relying on the VFS, check the hydration file is accessible
if fi, err := os.Stat(hydrationPath); err != nil || fi.Size() == 0 {
	// force re-hydration / litestream reset
}

n, err := v.ReadAt(buf, off)
if err != nil {
	var pe *os.PathError
	if errors.As(err, &pe) && errors.Is(pe.Err, syscall.ENOSPC) {
		// free disk space, then litestream reset
	}
}

When it happens

Trigger: A VFS read is issued against the hydrated file while it is closed, deleted, or on a failing disk; a ReadAt at an offset beyond a truncated file that surfaces a non-EOF OS error; file descriptor exhaustion causing the pread to fail.

Common situations: Disk full or hardware I/O errors on the node hosting the hydration file; the hydration temp file being removed by tmp cleaners while in use; running out of file descriptors under heavy SQLite load; permissions changed on the hydration directory mid-run.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• write updated page %d: %w
• write page %d to hydrated file: %w
• database does not exist: %w
• get initial size: %w
• get size after delete: %w

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/7892770301eae8ae. Report an issue: GitHub.