benbjohnson/litestream · error

enforce L%d retention: %w

Error message

enforce L%d retention: %w

What it means

Store.EnforceSnapshotRetention failed enforcing retention at a non-snapshot compaction level: db.EnforceRetentionByTXID(ctx, lvl.Level, minSnapshotTXID) errored while deleting LTX files below the minimum retained snapshot TXID. This cascades the snapshot retention decision down to levels L1+. The wrapped storage error is returned with %w including the level number.

Source

Thrown at store.go:850

// EnforceSnapshotRetention removes old snapshots by timestamp and then
// cleans up all lower levels based on minimum snapshot TXID.
func (s *Store) EnforceSnapshotRetention(ctx context.Context, db *DB) error {
	// Enforce retention for the snapshot level.
	minSnapshotTXID, err := db.EnforceSnapshotRetention(ctx, time.Now().Add(-s.SnapshotRetention))
	if err != nil {
		return fmt.Errorf("enforce snapshot retention: %w", err)
	}

	// We should also enforce retention for L0 on the same schedule as L1.
	for _, lvl := range s.levels {
		// Skip L0 since it is enforced on a more frequent basis.
		if lvl.Level == 0 {
			continue
		}

		if err := db.EnforceRetentionByTXID(ctx, lvl.Level, minSnapshotTXID); err != nil {
			return fmt.Errorf("enforce L%d retention: %w", lvl.Level, err)
		}
	}

	return nil
}

// ValidationResult holds the result of validating a replica's LTX files.
type ValidationResult struct {
	Valid  bool              // true if no errors found
	Errors []ValidationError // all errors found
}

// Validate checks LTX file consistency across all databases and levels.
// SnapshotLevel (9) is excluded since snapshots are not contiguous.
func (s *Store) Validate(ctx context.Context) (*ValidationResult, error) {
	result := &ValidationResult{Valid: true}

	s.mu.Lock()

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Check which level appears in the message (e.g. "enforce L1 retention") and test list/delete access on that storage prefix.
2. Grant delete permissions to the replica credentials; remove object-lock constraints that block cleanup.
3. Retry; retention enforcement runs on a schedule and is idempotent.
4. Verify replica contents are intact (no manually deleted level prefixes) — a missing level can break iteration.

Example fix

// before
if err := db.EnforceRetentionByTXID(ctx, lvl.Level, minSnapshotTXID); err != nil {
    return fmt.Errorf("enforce L%d retention: %w", lvl.Level, err)
}
// after
if err := db.EnforceRetentionByTXID(ctx, lvl.Level, minSnapshotTXID); err != nil {
    if ctx.Err() != nil {
        return ctx.Err() // propagate shutdown/cancel instead of storage error
    }
    return fmt.Errorf("enforce L%d retention: %w", lvl.Level, err)
}

for _, lvl := range []int{1, 2} {
    itr, err := client.LTXFiles(ctx, lvl, 0, false)
    if err != nil {
        return fmt.Errorf("level %d not listable: %w", lvl, err)
    }
    itr.Close()
}

err := store.EnforceSnapshotRetention(ctx, db)
if err != nil {
    var lvl int
    if _, scanErr := fmt.Sscanf(err.Error(), "enforce L%d retention", &lvl); scanErr == nil {
        log.Printf("retention failed at level %d; check delete perms on that prefix", lvl)
    }
    return err
}

When it happens

Trigger: EnforceSnapshotRetention(ctx, db) where EnforceRetentionByTXID for any level in s.levels (L1, L2, ...) fails: LTXFiles listing error or delete failure on the replica backend for that level.

Common situations: IAM credentials lacking delete rights on level prefixes, provider immutability/object-lock settings, partial storage outage affecting specific prefixes, or network interruption during a multi-level prune.

Understand the failure class

Background: Permission denied / not authorized / 403 Forbidden: access-control rejections when the caller lacks the required role, grant, or ownership — this error's family across 18 libraries.

Related errors

• enforce snapshot retention: %w
• fetch dst level info: %w
• fetch src level info: %w
• validate level %d for %s: %w
• snapshot retention must be greater than 0

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/4816c11b7753bbb4. Report an issue: GitHub.