benbjohnson/litestream · error

remove ltx files: %w

Error message

remove ltx files: %w

What it means

Wraps a failure from Replica.Client.DeleteLTXFiles when removing snapshot-level LTX files marked for deletion during EnforceSnapshotRetention. The remote deletion failed; local cleanup still proceeds and the operation can be retried safely since deletions are idempotent.

Source

Thrown at db.go:3001

	// Use the last deleted snapshot's MaxTXID, rather than the first retained
	// snapshot's MaxTXID, to preserve lower-level files in an in-flight restore
	// plan. TestStore_EnforceSnapshotRetention_RetainsInFlightRestorePlanFiles
	// guards this conservative floor.
	for i, info := range snapshots {
		if slices.Contains(deleted, info) {
			continue
		}
		if i > 0 {
			minSnapshotTXID = snapshots[i-1].MaxTXID
		}
		break
	}

	// Remove files marked for deletion from remote storage (unless retention disabled).
	if !db.RetentionEnabled {
		db.Logger.Debug("skipping remote deletion (retention disabled)", "level", SnapshotLevel, "count", len(deleted))
	} else if err := db.Replica.Client.DeleteLTXFiles(ctx, deleted); err != nil {
		return 0, fmt.Errorf("remove ltx files: %w", err)
	}

	// Always clean up local files.
	for _, info := range deleted {
		localPath := db.LTXPath(SnapshotLevel, info.MinTXID, info.MaxTXID)
		db.Logger.Debug("deleting local ltx file", "level", SnapshotLevel, "minTXID", info.MinTXID, "maxTXID", info.MaxTXID, "path", localPath)

		if err := os.Remove(localPath); err != nil && !os.IsNotExist(err) {
			db.Logger.Error("failed to remove local ltx file", "path", localPath, "error", err)
		}
	}

	return minSnapshotTXID, nil
}

// EnforceL0RetentionByTime retains L0 files until they have been compacted into
// L1 and have existed for at least L0Retention.
func (db *DB) EnforceL0RetentionByTime(ctx context.Context) error {

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Grant delete permission on the replica bucket/prefix to the credentials in use.
2. Check for object lock, retention policy, or lifecycle rules blocking deletes.
3. Inspect the wrapped error for the specific object keys and delete/verify manually.
4. Retry; deletion is idempotent and partial completions are harmless.
5. If a storage lifecycle policy handles cleanup, set retention disabled to skip remote deletion.

Example fix

// before
if err := db.Replica.Client.DeleteLTXFiles(ctx, deleted); err != nil { return 0, fmt.Errorf("remove ltx files: %w", err) }
// after (caller retry)
err := db.Replica.Client.DeleteLTXFiles(ctx, deleted)
if err != nil && isTransient(err) { err = db.Replica.Client.DeleteLTXFiles(ctx, deleted) }
if err != nil { return 0, fmt.Errorf("remove ltx files: %w", err) }

// ensure credentials allow delete before enabling retention
// aws iam simulate-principal-policy --action-names s3:DeleteObject

if err := db.Replica.Client.DeleteLTXFiles(ctx, deleted); err != nil {
    if isTransient(err) { scheduleRetry() } else { logAndAlert(err) }
}

When it happens

Trigger: EnforceSnapshotRetention with RetentionEnabled=true calling DeleteLTXFiles on expired snapshots while the backend rejects deletes: permission denied on bucket, object-lock/immutability policies, or a transient network error mid-batch.

Common situations: S3 bucket with Object Lock or Glacier storage class blocking deletes; IAM policy lacking s3:DeleteObject; GCS retention policy; network drop during deletion.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• remove expired l0 files: %w
• remove ltx files: %w
• abs: cannot delete ltx file %q: %w
• abs: cannot list blobs: %w
• abs: cannot delete blob %q: %w

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/c96594a1c91ae96f. Report an issue: GitHub.