benbjohnson/litestream · error

remove ltx files

Error message

remove ltx files: %w

What it means

EnforceSnapshotRetention removes expired snapshot-level LTX files from remote storage. When the underlying replica client's DeleteLTXFiles call fails, the error is wrapped with 'remove ltx files: %w'. This is a remote storage deletion failure during snapshot retention enforcement, not a local file operation.

Solutions

1. Verify storage credentials and IAM permissions allow batch deletion of objects in the replica path
2. Retry the retention enforcement — the next monitor cycle re-computes the expired set
3. Check the wrapped error for provider-specific causes (throttling, access denied) and address that backend issue
4. Confirm the replica storage endpoint is reachable and not in a degraded state

Example fix

// before: failing silently or ignoring
if err := c.EnforceSnapshotRetention(ctx); err != nil {
    c.logger.Debug("retention failed", "err", err)
}
// after: surface and retry with backoff
if err := c.EnforceSnapshotRetention(ctx); err != nil {
    return fmt.Errorf("snapshot retention: %w", err)
}

// Go: verify storage access before enabling retention
iter, err := client.LTXFiles(ctx, ltx.SnapshotLevel, 0, false)
if err != nil { log.Fatal(err) }
iter.Close()

if err := c.EnforceSnapshotRetention(ctx); err != nil {
    var netErr net.Error
    if errors.As(err, &netErr) { /* retry with backoff */ }
    return fmt.Errorf("snapshot retention: %w", err)
}

When it happens

Trigger: Calling EnforceSnapshotRetention (directly or via monitorSnapshots) when RetentionEnabled is true and client.DeleteLTXFiles(ctx, deleted) returns an error — e.g. network outage, storage backend permissions revoked, expired credentials, or S3-compatible API 5xx during the batch delete.

Common situations: Expired or rotated cloud credentials mid-run; storage bucket policies denying s3:DeleteMultiObjectDelete; transient network partition to the replica storage; provider rate limiting on batch deletes.

Related errors

• fetch l0 files
• fetch l1 files
• fetch ltx files
• fetch ltx files
• remove expired l0 files

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/46691e9f2589e6dd. Report an issue: GitHub.

Appendix: source

Thrown at compactor.go:273

		if info.CreatedAt.Before(timestamp) {
			deleted = append(deleted, info)
			continue
		}

		if minSnapshotTXID == 0 || info.MaxTXID < minSnapshotTXID {
			minSnapshotTXID = info.MaxTXID
		}
	}

	if len(deleted) > 0 && deleted[len(deleted)-1] == lastInfo {
		deleted = deleted[:len(deleted)-1]
	}

	if !c.RetentionEnabled {
		c.logger.Debug("skipping remote deletion (retention disabled)", "level", SnapshotLevel, "count", len(deleted))
	} else if err := c.client.DeleteLTXFiles(ctx, deleted); err != nil {
		return 0, fmt.Errorf("remove ltx files: %w", err)
	}

	if c.LocalFileDeleter != nil {
		for _, info := range deleted {
			c.logger.Debug("deleting local ltx file",
				"level", SnapshotLevel,
				"minTXID", info.MinTXID,
				"maxTXID", info.MaxTXID)
			if err := c.LocalFileDeleter(SnapshotLevel, info.MinTXID, info.MaxTXID); err != nil {
				c.logger.Error("failed to remove local ltx file", "error", err)
			}
		}
	}

	return minSnapshotTXID, nil
}

// EnforceRetentionByTXID deletes files at the given level with maxTXID below the target.

View on GitHub (pinned to 4ed7a308f6)