benbjohnson/litestream · error

read header: %w

Error message

read header: %w

What it means

NewWALReaderWithOffset wraps any failure from readHeader() with 'read header: %w'. readHeader reads the 32-byte WAL header and validates magic, checksum, version, page size and salt fields. Most commonly the wrapped error is io.EOF, which readHeader returns when the header checksum does not match — typically because the WAL header was only partially written during a crashed checkpoint.

Source

Thrown at wal_reader.go:55

		return nil, err
	}
	return r, nil
}

// NewWALReaderWithOffset returns a new instance of WALReader at a given offset.
// Salt must match or else no frames will be returned. Checksum calculated from
// from previous page.
func NewWALReaderWithOffset(ctx context.Context, rd io.ReaderAt, offset int64, salt1, salt2 uint32, logger *slog.Logger) (*WALReader, error) {
	// Ensure we are not starting on the first page since we need to read the previous.
	if offset <= WALHeaderSize {
		return nil, fmt.Errorf("offset (%d) must be greater than the wal header size (%d)", offset, WALHeaderSize)
	}

	r := &WALReader{r: rd, logger: logger}

	// Read header to determine page size & byte order.
	if err := r.readHeader(); err != nil {
		return nil, fmt.Errorf("read header: %w", err)
	}

	// Load in salt in case the beginning of the file has been overwritten.
	r.salt1, r.salt2 = salt1, salt2

	// Ensure offset is positioned on a frame start.
	frameSize := int64(r.pageSize + WALFrameHeaderSize)
	if (offset-WALHeaderSize)%frameSize != 0 {
		return nil, fmt.Errorf("unaligned wal offset %d for page size %d", offset, r.pageSize)
	}
	r.frameN = int((offset - WALHeaderSize) / frameSize)

	// Read previous page to load checksum. Context errors are returned as-is
	// so callers don't mistake a cancellation for a frame mismatch.
	r.frameN--
	if _, _, err := r.readFrame(ctx, make([]byte, r.pageSize), false); err != nil {
		if ctx.Err() != nil {
			return nil, context.Cause(ctx)

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Verify the -wal file exists and is at least 32 bytes (ls -l path.wal); an empty/truncated WAL cannot be read
2. Treat io.EOF wrapped by this error as 'no valid WAL yet' and skip/wait — SQLite will rewrite the header on the next write
3. Confirm only one process is writing the database (a competing checkpoint can truncate the header mid-read)
4. If the file is consistently unreadable, restore the database from the latest replica backup and let Litestream re-replicate

Example fix

// before: crashing on truncated WAL
r, err := NewWALReaderWithOffset(ctx, f, offset, logger)
if err != nil { return err }
// after: tolerate not-yet-valid WAL headers
r, err := NewWALReaderWithOffset(ctx, f, offset, logger)
if errors.Is(err, io.EOF) { return nil // WAL header not written yet; retry later }
if err != nil { return err }

fi, err := os.Stat(walPath)
if err != nil { return err }
if fi.Size() < 32 {
    return fmt.Errorf("wal %s too small (%d bytes); not yet written or truncated", walPath, fi.Size())
}

func hasValidWALSize(fi os.FileInfo) bool { return fi != nil && fi.Size() >= 32 }

r, err := NewWALReaderWithOffset(ctx, f, off, logger)
if errors.Is(err, io.EOF) {
    // partial/absent WAL header — retry later
    return nil
} else if err != nil {
    return fmt.Errorf("open wal: %w", err)
}

When it happens

Trigger: Calling NewWALReaderWithOffset (directly or via sync) on a WAL file that is shorter than 32 bytes, is empty, or whose 32-byte header fails the checksum verification (returning the wrapped io.EOF).

Common situations: A Litestream process or SQLite checkpoint crashed mid-write leaving a zero-length or truncated -wal file; the WAL file was recreated/truncated by another tool; reading a WAL snapshot copied before the header was flushed.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

• set synchronous: %w
• checkpoint: %w
• checkpoint failed: %w
• sync database %s: %w
• enable wal failed, mode=%q

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/63210e4094f842a0. Report an issue: GitHub.