benbjohnson/litestream · error

iterate remote files

Error message

iterate remote files: %w

What it means

After enumerating remote LTX files to compute the remote position, the iterator must be closed; closing can flush/propagate backend errors (e.g. a failed final listing page or checksum error). This error wraps itr.Close() failure from the conflict check, so sync is aborted even though enumeration appeared to succeed.

Solutions

1. Retry the sync; if the error is transient the next LTXFiles listing usually succeeds.
2. Inspect the wrapped error for HTTP/connection details and fix endpoint/proxy settings accordingly.
3. Test listing stability against the replica with the litestream ltx command; if listings repeatedly truncate, check the gateway (MinIO/version) health.
4. Update Litestream / replica client SDK versions if the provider changed pagination behavior.

if err := db.Sync(ctx); err != nil {
    if strings.Contains(err.Error(), "iterate remote files") {
        // listing closed with backend error; retry after short delay
        time.Sleep(2 * time.Second)
        return db.Sync(ctx)
    }
    return err
}

When it happens

Trigger: itr.Close() on the LTX file iterator returns an error — typically the underlying object-store listing failed partway (truncated response, connection reset) and the error surfaces on Close.

Common situations: Connection resets to S3-compatible endpoints mid-listing; proxies cutting long-lived responses; provider inconsistencies during bucket failover; buggy S3 gateways (MinIO) returning malformed truncated listings.

Related errors

• check remote position
• close level ltx iterator
• apply WAL segments
• cannot determine L max ltx file for
• close iterator

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/7fd1e160a29b63e2. Report an issue: GitHub.

Appendix: source

Thrown at vfs.go:2080

// checkForConflict checks if the remote has newer transactions than expected.
// Must be called with f.mu held.
func (f *VFSFile) checkForConflict(ctx context.Context) error {
	// Get latest remote position
	itr, err := f.client.LTXFiles(ctx, 0, f.expectedTXID, false)
	if err != nil {
		return fmt.Errorf("check remote position: %w", err)
	}
	defer itr.Close()

	var remoteTXID ltx.TXID
	for itr.Next() {
		info := itr.Item()
		if info.MaxTXID > remoteTXID {
			remoteTXID = info.MaxTXID
		}
	}
	if err := itr.Close(); err != nil {
		return fmt.Errorf("iterate remote files: %w", err)
	}

	// If remote has advanced beyond our expected position, we have a conflict
	if remoteTXID > f.expectedTXID {
		f.logger.Warn("conflict detected",
			"expected", f.expectedTXID,
			"remote", remoteTXID)
		return fmt.Errorf("%w: expected TXID %d but remote has %d",
			ErrConflict, f.expectedTXID, remoteTXID)
	}

	return nil
}

// createLTXFromDirty creates an LTX file from dirty pages.
// Returns a streaming reader for the LTX data using io.Pipe to avoid loading
// all data into memory at once.
// Must be called with f.mu held.

View on GitHub (pinned to 4ed7a308f6)