benbjohnson/litestream · error

remove stale hydration meta: %w

Error message

remove stale hydration meta: %w

What it means

For a persistent hydrator whose metadata is stale or whose hydration file is missing, Init removes the leftover meta file before recreating the hydration file. This error wraps os.Remove failing on the meta file (any error other than NotExist). It indicates stale hydration state could not be cleaned up, so Init aborts rather than continue with inconsistent on-disk state.

Source

Thrown at vfs.go:697

func (h *Hydrator) Init() error {
	if err := os.MkdirAll(filepath.Dir(h.path), 0755); err != nil {
		return fmt.Errorf("create hydration directory: %w", err)
	}

	if h.persistent {
		if txid, err := h.loadMeta(); err == nil {
			if _, statErr := os.Stat(h.path); statErr == nil {
				file, err := os.OpenFile(h.path, os.O_RDWR, 0600)
				if err != nil {
					return fmt.Errorf("open persistent hydration file: %w", err)
				}
				h.file = file
				h.txid = txid
				return nil
			}
		}
		if err := os.Remove(h.metaPath()); err != nil && !os.IsNotExist(err) {
			return fmt.Errorf("remove stale hydration meta: %w", err)
		}
	}

	file, err := os.OpenFile(h.path, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0600)
	if err != nil {
		return fmt.Errorf("create hydration file: %w", err)
	}
	h.file = file
	return nil
}

// Complete returns true if hydration has completed.
func (h *Hydrator) Complete() bool {
	return h.complete.Load()
}

// SetComplete marks hydration as complete.
func (h *Hydrator) SetComplete() {

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Grant the process user write+execute permission on the hydration directory so it can unlink files.
2. Remove the immutable attribute if set (chattr -i <meta-file>) or check for read-only mounts (mount | grep ro).
3. Manually delete the stale meta file as an administrator and retry Init.
4. Nuke the whole hydration directory and let Init rebuild it if state is disposable.

Example fix

// before
$ lsattr hydration.db.meta
----i--------- hydration.db.meta  # immutable

// after
$ sudo chattr -i hydration.db.meta && rm hydration.db.meta  # then retry Init

mp := hydrator.MetaPath()
dir := filepath.Dir(mp)
if err := unix.Access(dir, unix.W_OK); err != nil {
    return fmt.Errorf("cannot unlink in %q: %w", dir, err)
}

if err := hydrator.Init(); err != nil && strings.Contains(err.Error(), "remove stale hydration meta") {
    // check dir perms / immutable flags / ro mount, clean manually, retry
}

When it happens

Trigger: Persistent hydration Init where loadMeta fails or the hydration file is absent, and deleting the meta file fails due to directory permissions, immutable flag, or I/O error on the containing directory.

Common situations: Directory owned by a different user after a container image update; chattr +i applied by hardening scripts; read-only remount of the volume mid-run.

Understand the failure class

Background: Permission denied / not authorized / 403 Forbidden: access-control rejections when the caller lacks the required role, grant, or ownership — this error's family across 18 libraries.

Related errors

• create hydration directory: %w
• open persistent hydration file: %w
• create hydration file: %w
• cannot remove tmp files: %w
• open wal file: %w

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/900c08b295c29885. Report an issue: GitHub.