benbjohnson/litestream · error

close iterator: %w

Error message

close iterator: %w

What it means

VerifyLevelConsistency finished walking a compaction level's LTX files and closing the iterator returned a non-nil error; the check fails with 'close iterator: <err>'. This guards the invariant that a level's files chain correctly (prev TXID linkage), and a close error means the underlying storage client couldn't cleanly finish the listing, so verification results cannot be trusted.

Source

Thrown at compactor.go:229

			continue
		}

		// Check for TXID contiguity: prev.MaxTXID + 1 should equal curr.MinTXID
		expectedMinTXID := prevInfo.MaxTXID + 1
		if info.MinTXID != expectedMinTXID {
			if info.MinTXID > expectedMinTXID {
				return fmt.Errorf("TXID gap detected: prev.MaxTXID=%s, next.MinTXID=%s (expected %s)",
					prevInfo.MaxTXID, info.MinTXID, expectedMinTXID)
			}
			return fmt.Errorf("TXID overlap detected: prev.MaxTXID=%s, next.MinTXID=%s",
				prevInfo.MaxTXID, info.MinTXID)
		}

		prevInfo = info
	}

	if err := itr.Close(); err != nil {
		return fmt.Errorf("close iterator: %w", err)
	}

	return nil
}

// EnforceSnapshotRetention enforces retention of snapshot level files by timestamp.
// Files older than the retention duration are deleted (except the newest is always kept).
// Returns the minimum snapshot TXID still retained (useful for cascading retention to lower levels).
func (c *Compactor) EnforceSnapshotRetention(ctx context.Context, retention time.Duration) (ltx.TXID, error) {
	timestamp := time.Now().Add(-retention)
	c.logger.Debug("enforcing snapshot retention", "timestamp", timestamp)

	itr, err := c.client.LTXFiles(ctx, SnapshotLevel, 0, false)
	if err != nil {
		return 0, fmt.Errorf("fetch ltx files: %w", err)
	}
	defer itr.Close()

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Inspect the wrapped inner error — network issues usually resolve on retry of the compaction
2. Verify storage backend connectivity/credentials (S3 etc.)
3. Check that a custom ReplicaClient's LTXFiles iterator Close is implemented correctly (must release resources, return nil on clean close)
4. If caused by context cancelation, ensure the parent context has enough time/budget for compaction

// before compaction, sanity-check storage reachability:
itr, err := client.LTXFiles(ctx, level, 0, false)
if err != nil { return err }
itr.Close()

if err := compactor.Compact(ctx); err != nil {
    var retryable bool
    if strings.Contains(err.Error(), "close iterator") && errors.Is(ctx.Err(), nil) {
        retryable = true // transient storage error — backoff and retry
    }
}

When it happens

Trigger: Called from Compact after iterating level files via c.client.LTXFiles; the storage backend's iterator Close fails — e.g. S3 listing session teardown error, network reset during final pagination flush, or context canceled while closing.

Common situations: Flaky network to the replica storage during compaction; context deadline/cancelation from the caller of Compact; misbehaving custom ReplicaClient whose Close implementation errors.

Related errors

• fetch ltx files: %w
• read ltx page index: %w
• read ltx page frame: %w
• fetch updated page %d: %w
• fetch page %d: %w

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/9a400d4e45ff2fe7. Report an issue: GitHub.