benbjohnson/litestream · error

iterate remote files: %w

Error message

iterate remote files: %w

What it means

After enumerating remote LTX files to compute the remote position, the iterator must be closed; closing can flush/propagate backend errors (e.g. a failed final listing page or checksum error). This error wraps itr.Close() failure from the conflict check, so sync is aborted even though enumeration appeared to succeed.

Source

Thrown at vfs.go:2080

// checkForConflict checks if the remote has newer transactions than expected.
// Must be called with f.mu held.
func (f *VFSFile) checkForConflict(ctx context.Context) error {
	// Get latest remote position
	itr, err := f.client.LTXFiles(ctx, 0, f.expectedTXID, false)
	if err != nil {
		return fmt.Errorf("check remote position: %w", err)
	}
	defer itr.Close()

	var remoteTXID ltx.TXID
	for itr.Next() {
		info := itr.Item()
		if info.MaxTXID > remoteTXID {
			remoteTXID = info.MaxTXID
		}
	}
	if err := itr.Close(); err != nil {
		return fmt.Errorf("iterate remote files: %w", err)
	}

	// If remote has advanced beyond our expected position, we have a conflict
	if remoteTXID > f.expectedTXID {
		f.logger.Warn("conflict detected",
			"expected", f.expectedTXID,
			"remote", remoteTXID)
		return fmt.Errorf("%w: expected TXID %d but remote has %d",
			ErrConflict, f.expectedTXID, remoteTXID)
	}

	return nil
}

// createLTXFromDirty creates an LTX file from dirty pages.
// Returns a streaming reader for the LTX data using io.Pipe to avoid loading
// all data into memory at once.
// Must be called with f.mu held.

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Retry the sync; if the error is transient the next LTXFiles listing usually succeeds.
2. Inspect the wrapped error for HTTP/connection details and fix endpoint/proxy settings accordingly.
3. Test listing stability against the replica with the litestream ltx command; if listings repeatedly truncate, check the gateway (MinIO/version) health.
4. Update Litestream / replica client SDK versions if the provider changed pagination behavior.

if err := db.Sync(ctx); err != nil {
    if strings.Contains(err.Error(), "iterate remote files") {
        // listing closed with backend error; retry after short delay
        time.Sleep(2 * time.Second)
        return db.Sync(ctx)
    }
    return err
}

When it happens

Trigger: itr.Close() on the LTX file iterator returns an error — typically the underlying object-store listing failed partway (truncated response, connection reset) and the error surfaces on Close.

Common situations: Connection resets to S3-compatible endpoints mid-listing; proxies cutting long-lived responses; provider inconsistencies during bucket failover; buggy S3 gateways (MinIO) returning malformed truncated listings.

Related errors

• close level %d ltx iterator: %w
• check remote position: %w
• sync interval must be greater than 0
• close l1 iterator: %w
• replica sync: %w

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/7fd1e160a29b63e2. Report an issue: GitHub.