benbjohnson/litestream · error

list level ltx files

Error message

list level %d ltx files: %w

What it means

fillFollowGap, which tries to bridge a TXID gap in level-0 files by scanning compaction levels 1..SnapshotLevel-1, failed while listing the LTX files for a level via the storage client (r.Client.LTXFiles). The wrapped error comes from the storage backend, not from local files.

Solutions

1. Verify storage credentials and permissions for listing objects in the bucket/container
2. Check network connectivity and proxy settings to the storage endpoint
3. Confirm bucket/region/container configuration in the litestream config file
4. Check for storage-side rate limits (429s) and add backoff/reduce polling frequency

Example fix

// before: missing s3:ListBucket permission
// policy lacks s3:ListBucket on the bucket
// after: grant it
{ "Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject", "s3:ListBucket"], "Resource": ["arn:aws:s3:::my-bucket", "arn:aws:s3:::my-bucket/*"] }

// preflight: can we list the replica path?
itr, err := client.LTXFiles(ctx, 1, 0, false)
if err != nil {
    return fmt.Errorf("storage preflight failed (check credentials/bucket): %w", err)
}
itr.Close()

if err := r.applyNewLTXFiles(ctx, f, pageSize); err != nil {
    var storageErr *StorageError
    if errors.As(err, &storageErr) || strings.Contains(err.Error(), "list level") {
        // transient listing failure: retry with backoff
        return retryWithBackoff(3, func() error { return r.applyNewLTXFiles(ctx, f, pageSize) })
    }
    return err
}

When it happens

Trigger: r.Client.LTXFiles(ctx, level, 0, false) returns an error for levels 1-3: storage API failure listing objects, expired/invalid credentials, network failure, or a misconfigured bucket/container.

Common situations: S3/GCS/Azure credential expiry or permission loss, rate limiting on ListObjects, region or bucket name misconfiguration, network outage during replica follow mode.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

• cannot determine L max ltx file for
• check remote position
• close level ltx iterator
• fetch ltx files
• fetch updated page

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/3fb280f7419d43e3. Report an issue: GitHub.

Appendix: source

Thrown at replica.go:1010

		}
	}

	if err := dec.Close(); err != nil {
		return fmt.Errorf("close decoder: %w", err)
	}

	return f.Sync()
}

// fillFollowGap attempts to bridge a gap in level 0 files by searching
// higher compaction levels for a file that covers the missing TXID range.
func (r *Replica) fillFollowGap(ctx context.Context, f *os.File, afterTXID ltx.TXID, gapMinTXID ltx.TXID, pageSize uint32) (ltx.TXID, error) {
	currentTXID := afterTXID

	for level := 1; level < SnapshotLevel; level++ {
		itr, err := r.Client.LTXFiles(ctx, level, 0, false)
		if err != nil {
			return currentTXID, fmt.Errorf("list level %d ltx files: %w", level, err)
		}
		closeLevel := func(retErr error) (ltx.TXID, error) {
			if closeErr := itr.Close(); closeErr != nil {
				closeErr = fmt.Errorf("close level %d ltx iterator: %w", level, closeErr)
				if retErr != nil {
					return currentTXID, errors.Join(retErr, closeErr)
				}
				return currentTXID, closeErr
			}
			return currentTXID, retErr
		}

		for itr.Next() {
			info := itr.Item()

			// Skip if there's a gap at this level too.
			if info.MinTXID > currentTXID+1 {
				break

View on GitHub (pinned to 4ed7a308f6)