benbjohnson/litestream · error

decode database: %w

Error message

decode database: %w

What it means

After compacting LTX readers through an io.Pipe, Hydrator.Restore decodes the compacted stream directly into the hydration file with dec.DecodeDatabaseTo(h.file). This error wraps any failure while streaming the decoded database: compaction errors propagated through the pipe, checksum verification failures, or write errors on the local hydration file.

Source

Thrown at vfs.go:803

	// Compact and decode using io.Pipe pattern
	pr, pw := io.Pipe()
	c, err := ltx.NewCompactor(pw, rdrs)
	if err != nil {
		return fmt.Errorf("new ltx compactor: %w", err)
	}
	c.HeaderFlags = ltx.HeaderFlagNoChecksum
	h.compactor = c

	go func() {
		_ = pw.CloseWithError(c.Compact(ctx))
	}()

	h.mu.Lock()
	defer h.mu.Unlock()

	dec := ltx.NewDecoder(pr)
	if err := dec.DecodeDatabaseTo(h.file); err != nil {
		return fmt.Errorf("decode database: %w", err)
	}

	h.txid = infos[len(infos)-1].MaxTXID
	return nil
}

// CatchUp applies updates from LTX files between fromTXID and toTXID.
func (h *Hydrator) CatchUp(ctx context.Context, fromTXID, toTXID ltx.TXID) error {
	h.logger.Debug("catching up hydration", "from", fromTXID, "to", toTXID)

	// Fetch LTX files from fromTXID+1 to toTXID
	itr, err := h.client.LTXFiles(ctx, 0, fromTXID+1, false)
	if err != nil {
		return fmt.Errorf("list ltx files for catch-up: %w", err)
	}
	defer itr.Close()

	for itr.Next() {

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Inspect the wrapped cause: if it came through the pipe, the real failure is c.Compact's error (upstream read/checksum) — re-fetch or re-verify the replica files.
2. Check free disk space and write permissions on the directory holding the hydration file; retry after clearing space.
3. Delete the partial hydration file and re-run Restore from scratch; use `litestream reset` if local LTX state is corrupt.
4. Confirm the replica's page size matches the local database configuration; a mismatched -page-size between backup and restore causes decode failure.

Example fix

// before
if err := dec.DecodeDatabaseTo(h.file); err != nil {
    return fmt.Errorf("decode database: %w", err)
}
// after
if err := dec.DecodeDatabaseTo(h.file); err != nil {
    _ = h.file.Close()
    _ = os.Remove(h.path) // drop partial hydration file
    return fmt.Errorf("decode database: %w", err)
}

// verify free space before decoding
if st, err := os.Stat(filepath.Dir(h.path)); err == nil {
    _ = st // ensure mount exists and is writable via a probe write
}

if err := dec.DecodeDatabaseTo(h.file); err != nil {
    os.Remove(h.path) // drop partial hydration file
    return fmt.Errorf("decode database: %w", err)
}

When it happens

Trigger: Calling Restore where the compacted stream is truncated or its checksums don't match (corrupt replica objects, interrupted download surfaced via pw.CloseWithError), the hydration file's disk is full or unwritable, or page size in the LTX header doesn't match h.pageSize.

Common situations: Flaky object storage dropping bytes mid-stream; disk quota exceeded on the node holding the hydration file; replica files corrupted by a partial upload; restoring across litestream versions with differing header flags (HeaderFlagNoChecksum vs checksummed).

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

• verify
• invalid replica, checksum mismatch
• decode
• decode ltx page: %w
• created at: %w

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/2847e164ffa352ae. Report an issue: GitHub.