thanos-io/thanos · error

Found overlap or error during sync, cannot upload compacted…

Error message

Found overlap or error during sync, cannot upload compacted block, details: %v

What it means

Returned by Sync when the overlap checker reports that a compacted block (Compaction.Level > 1) overlaps already-shipped blocks and out-of-order uploads are disabled. Sync aborts immediately so the conflicting block is never uploaded, protecting the non-overlapping block invariant in the bucket.

Solutions

1. Read the embedded overlap details, identify overlapping ULIDs, and delete the redundant block (local + bucket via `thanos tools bucket rm`).
2. Re-compact so overlapping blocks merge into one non-overlapping block, then re-run Sync.
3. Enable --shipper.allow-out-of-order-uploads if your deployment deliberately permits overlaps.
4. Ensure only one sidecar ships a given TSDB directory and snapshots aren't re-imported over live data.

Example fix

// before
// Found overlap or error during sync, cannot upload compacted block, details: shipping compacted block 01ARZ... is blocked; overlap spotted: [...]
// after
// thanos tools bucket rm --objstore.config-file=bucket.yaml --id=<duplicate-ULID>
// # or: thanos sidecar ... --shipper.allow-out-of-order-uploads

if blockMeta.Compaction.Level > 1 && !allowOutOfOrder {
    if err := checker.IsOverlapping(ctx, blockMeta); err != nil {
        return fmt.Errorf("skip upload of %s: %w", blockMeta.ULID, err)
    }
}

if err := shipper.Sync(ctx); err != nil && strings.Contains(err.Error(), "overlap") {
    // parse ULIDs from details, resolve duplicates (bucket rm / recompact), then re-sync
    log.Printf("overlap conflict requiring manual resolution: %v", err)
}

When it happens

Trigger: Sync iterates blocks; for a block with Compaction.Level > 1 and s.allowOutOfOrderUploads == false, checker.IsOverlapping returns the overlap error from [901], which Sync re-wraps with this message.

Common situations: Backfilled/restored TSDB data overlapping shipped blocks; Prometheus running with overlapping compaction (misconfigured or third-party fork); two sidecars shipping the same TSDB with divergent compaction; snapshot-based recovery producing duplicate ranges.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• shipping compacted block
• level is bigger then default set of
• get compaction levels
• create compactor
• create bucket compactor

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/b82d373d51502e6d. Report an issue: GitHub.

Appendix: source

Thrown at pkg/shipper/shipper.go:423

			if !s.uploadCompacted {
				continue
			}
		}

		// Check against bucket if the meta file for this block exists.
		ok, err := s.bucket.Exists(ctx, path.Join(m.ULID.String(), block.MetaFilename))
		if err != nil {
			return uploaded, errors.Wrap(err, "check exists")
		}
		if ok {
			meta.Uploaded = append(meta.Uploaded, m.ULID)
			continue
		}

		// Skip overlap check if out of order uploads is enabled.
		if m.Compaction.Level > 1 && !s.allowOutOfOrderUploads {
			if err := checker.IsOverlapping(ctx, m.BlockMeta); err != nil {
				return uploaded, errors.Errorf("Found overlap or error during sync, cannot upload compacted block, details: %v", err)
			}
		}

		if err := s.upload(ctx, m); err != nil {
			if !s.allowOutOfOrderUploads {
				return uploaded, errors.Wrapf(err, "upload %v", m.ULID)
			}

			// No error returned, just log line. This is because we want other blocks to be uploaded even
			// though this one failed. It will be retried on second Sync iteration.
			level.Error(s.logger).Log("msg", "shipping failed", "block", m.ULID, "err", err)
			uploadErrs++
			continue
		}
		meta.Uploaded = append(meta.Uploaded, m.ULID)
		uploaded++
		s.metrics.uploads.Inc()
	}

View on GitHub (pinned to 35b8b99117)