benbjohnson/litestream · error

remove expired l0 files: %w

Error message

remove expired l0 files: %w

What it means

After selecting expired L0 files, EnforceL0Retention deletes them via client.DeleteLTXFiles; failure is wrapped as 'remove expired l0 files: %w'. This blocks L0 cleanup so transactional LTX files will accumulate in remote storage until deletion succeeds.

Source

Thrown at compactor.go:409

		}

		if info.MaxTXID <= maxL1TXID {
			deleted = append(deleted, info)
		}
	}

	if processedAll && len(deleted) > 0 && lastInfo != nil && deleted[len(deleted)-1] == lastInfo {
		deleted = deleted[:len(deleted)-1]
	}

	if len(deleted) == 0 {
		return nil
	}

	if !c.RetentionEnabled {
		c.logger.Debug("skipping remote deletion (retention disabled)", "level", 0, "count", len(deleted))
	} else if err := c.client.DeleteLTXFiles(ctx, deleted); err != nil {
		return fmt.Errorf("remove expired l0 files: %w", err)
	}

	if c.LocalFileDeleter != nil {
		for _, info := range deleted {
			c.logger.Debug("deleting expired local l0 file",
				"minTXID", info.MinTXID,
				"maxTXID", info.MaxTXID)
			if err := c.LocalFileDeleter(0, info.MinTXID, info.MaxTXID); err != nil {
				c.logger.Error("failed to remove local l0 file", "error", err)
			}
		}
	}

	c.logger.Info("l0 retention enforced", "deleted_count", len(deleted), "max_l1_txid", maxL1TXID)

	return nil
}

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Check the wrapped error — for throttling, ensure fewer files accumulate (fix L1 compaction) or use a provider supporting large batch deletes
2. Remove competing lifecycle rules or concurrent retention enforcers
3. Grant delete permissions on the replica prefix
4. Retry; the next monitorL0Retention cycle recomputes the expired set
5. If deletion must be handled externally, set RetentionEnabled=false and manage cleanup via cloud lifecycle policy

// Bound the batch size / ensure retention cadence so deletes stay small
if len(expired) > maxBatch { expired = expired[:maxBatch] }

if err := c.EnforceL0Retention(ctx, retention); err != nil {
    var apiErr *provider.BatchDeleteError
    if errors.As(err, &apiErr) && apiErr.Throttled { /* retry later */ }
}

When it happens

Trigger: RetentionEnabled is true and the batch delete of expired L0 files errors — provider throttling on large batches, missing delete permission, network failure, or objects already deleted by a competing process.

Common situations: Thousands of accumulated L0 files making batch deletes hit provider limits; lifecycle rules already deleting files (conflict); IAM policy lacking multi-object delete; sustained network outage.

Related errors

• remove ltx files: %w
• fetch l1 files: %w
• fetch l0 files: %w
• remove ltx files: %w
• remove expired l0 files: %w

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/fe6d1ddf49d2dc1c. Report an issue: GitHub.