thanos-io/thanos · critical

create meta fetcher

Error message

create meta fetcher

What it means

During compact/replicate setup, block.NewBaseFetcher creates the fetcher that lists bucket objects and downloads block metadata. This error wraps its initialization failure — typically a problem listing or reading meta.json files from the object store, or invalid fetcher options. It occurs before any syncing starts.

Solutions

1. Read the wrapped inner error — it usually names the exact object-store call that failed.
2. Verify the bucket is reachable and credentials allow listing: `thanos tools bucket ls <bucket>`.
3. Check IAM permissions: ListObjects/Get must be allowed for the configured prefix.
4. Validate --objstore.config-file (endpoint, region, bucket name) for typos.
5. Check network/proxy/DNS connectivity from the host to the storage endpoint.

Example fix

// before: bucket name typo in config
confContentYaml = []byte("type: S3\nbucket: thano-blocks")
// after
confContentYaml = []byte("type: S3\nbucket: thanos-blocks\nendpoint: s3.amazonaws.com\nregion: us-east-1")

// Pre-flight: confirm the bucket is listable with the same config
ls := client.NewBucket(...) // or simply run:
// thanos tools bucket ls --objstore.config-file=b.yml | head -1

baseMetaFetcher, err := block.NewBaseFetcher(...)
if err != nil {
    if isTransientStorageError(errors.Unwrap(err)) {
        time.Sleep(retryBackoff)
        baseMetaFetcher, err = block.NewBaseFetcher(...) // retry
    }
}

When it happens

Trigger: Running a compact/replicate command where NewBaseFetcher fails at cmd/thanos/tools_bucket.go:875 — usually the initial ListObjects call fails (bad bucket client config, missing permissions) or the fetcher's concurrency/registry options are invalid.

Common situations: Bucket credentials lacking s3:ListBucket permission, wrong bucket name/endpoint in the objstore config, network/proxy blocking the storage API, or a Thanos version mismatch making the fetcher reject the bucket layout.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

• sync blocks
• filter blocks marked for deletion
• create meta fetcher
• create syncer
• compaction

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/245ee02a41b32d2c. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/tools_bucket.go:875

		g.Add(func() error { return nil }, func(error) {})

		stubCounter := promauto.With(nil).NewCounter(prometheus.CounterOpts{})

		// While fetching blocks, we filter out blocks that were marked for deletion by using IgnoreDeletionMarkFilter.
		// The delay of deleteDelay/2 is added to ensure we fetch blocks that are meant to be deleted but do not have a replacement yet.
		// This is to make sure compactor will not accidentally perform compactions with gap instead.
		ignoreDeletionMarkFilter := block.NewIgnoreDeletionMarkFilter(logger, insBkt, tbc.deleteDelay/2, tbc.blockSyncConcurrency)
		duplicateBlocksFilter := block.NewDeduplicateFilter(tbc.blockSyncConcurrency)
		blocksCleaner := compact.NewBlocksCleaner(logger, insBkt, ignoreDeletionMarkFilter, tbc.deleteDelay, stubCounter, stubCounter)

		ctx := context.Background()

		var sy *compact.Syncer
		{
			baseBlockIDsFetcher := block.NewConcurrentLister(logger, insBkt)
			baseMetaFetcher, err := block.NewBaseFetcher(logger, tbc.blockSyncConcurrency, insBkt, baseBlockIDsFetcher, "", extprom.WrapRegistererWithPrefix(extpromPrefix, reg))
			if err != nil {
				return errors.Wrap(err, "create meta fetcher")
			}
			cf := baseMetaFetcher.NewMetaFetcher(
				extprom.WrapRegistererWithPrefix(extpromPrefix, reg), []block.MetadataFilter{
					block.NewLabelShardedMetaFilter(relabelConfig),
					block.NewConsistencyDelayMetaFilter(logger, tbc.consistencyDelay, extprom.WrapRegistererWithPrefix(extpromPrefix, reg)),
					ignoreDeletionMarkFilter,
					duplicateBlocksFilter,
				},
			)
			sy, err = compact.NewMetaSyncer(
				logger,
				reg,
				insBkt,
				cf,
				duplicateBlocksFilter,
				ignoreDeletionMarkFilter,
				stubCounter,
				stubCounter,

View on GitHub (pinned to 35b8b99117)