thanos-io/thanos · error

check exists in bucket

Error message

check exists %s in bucket

What it means

MarkForNoCompact first checks whether the no-compact mark file (<blockID>/no-compact-mark.json) already exists in the bucket; this error wraps a failed bkt.Exists call. It indicates the object storage existence check itself failed (network/auth error), not that the mark exists.

Solutions

1. Retry the operation; Exists failures are usually transient object-store errors.
2. Verify bucket credentials and that the bucket exists and is accessible.
3. Check network connectivity/egress to the object storage endpoint.
4. Check ctx cancellation if timeouts occur consistently.

Example fix

// before
noCompactMarkExists, err := bkt.Exists(ctx, m)
if err != nil {
	return errors.Wrapf(err, "check exists %s in bucket", m)
}
// after
noCompactMarkExists, err := bkt.Exists(ctx, m)
if err != nil {
	if ctx.Err() != nil {
		return errors.Wrapf(ctx.Err(), "check exists %s in bucket: context cancelled", m)
	}
	return errors.Wrapf(err, "check exists %s in bucket", m)
}

if err := ctx.Err(); err != nil { return err }
if err := bkt.Iter(ctx, "", func(string) error { return nil }); err != nil {
	return fmt.Errorf("bucket not reachable before mark: %w", err)
}

func bucketReachable(ctx context.Context, bkt objstore.Bucket) bool {
	return bkt.Iter(ctx, "", func(string) error { return nil }) == nil
}

err := block.MarkForNoCompact(ctx, logger, bkt, id, reason, details, noCompactMarked)
if err != nil {
	if ctx.Err() != nil { return err }
	// transient object-store failure: retry with backoff
	return retryWithBackoff(ctx, 3, time.Second, func() error {
		return block.MarkForNoCompact(ctx, logger, bkt, id, reason, details, noCompactMarked)
	})
}

When it happens

Trigger: bkt.Exists(ctx, id/no-compact-mark.json) returns an error — object storage unreachable, credentials invalid, bucket missing, or context cancelled during the HEAD request.

Common situations: Temporary S3/GCS outage; expired credentials (IAM role refresh failure); wrong bucket name in config; context deadline exceeded while marking a block after compaction planning.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• read meta.json for block
• get meta file
• read meta file
• get TOC from object storage of
• get header from object storage of

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/c03ff29ee231a7cf. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/block.go:386

	metaFile, err := os.Stat(filepath.Join(blockDir, MetaFilename))
	if err != nil {
		return nil, errors.Wrapf(err, "stat %v", filepath.Join(blockDir, MetaFilename))
	}
	res = append(res, metadata.File{RelPath: metaFile.Name()})

	sort.Slice(res, func(i, j int) bool {
		return strings.Compare(res[i].RelPath, res[j].RelPath) < 0
	})
	return res, err
}

// MarkForNoCompact creates a file which marks block to be not compacted.
func MarkForNoCompact(ctx context.Context, logger log.Logger, bkt objstore.Bucket, id ulid.ULID, reason metadata.NoCompactReason, details string, markedForNoCompact prometheus.Counter) error {
	m := path.Join(id.String(), metadata.NoCompactMarkFilename)
	noCompactMarkExists, err := bkt.Exists(ctx, m)
	if err != nil {
		return errors.Wrapf(err, "check exists %s in bucket", m)
	}
	if noCompactMarkExists {
		level.Warn(logger).Log("msg", "requested to mark for no compaction, but file already exists; this should not happen; investigate", "err", errors.Errorf("file %s already exists in bucket", m))
		return nil
	}

	noCompactMark, err := json.Marshal(metadata.NoCompactMark{
		ID:      id,
		Version: metadata.NoCompactMarkVersion1,

		NoCompactTime: time.Now().Unix(),
		Reason:        reason,
		Details:       details,
	})
	if err != nil {
		return errors.Wrap(err, "json encode no compact mark")
	}

View on GitHub (pinned to 35b8b99117)