thanos-io/thanos · error

create meta fetcher

Error message

create meta fetcher

What it means

Wraps an error from block.NewBaseFetcher, which sets up the metadata fetcher that downloads and validates block meta.json files into conf.dataDir. Typical underlying causes are failure to create the local data dir (permissions, disk full) or invalid concurrency settings.

Solutions

1. Inspect the wrapped cause in logs and fix the underlying dataDir problem (permissions, space).
2. Ensure conf.dataDir is a writable directory path.
3. Check disk usage/free space on the volume backing dataDir.

Example fix

// before
chown -R root:root /data  # dataDir not writable by thanos user
// after
chown -R thanos:thanos /data && chmod u+rwX /data

if fi, err := os.Stat(dataDir); err != nil || !fi.IsDir() {
    return fmt.Errorf("data-dir %q must be an existing directory", dataDir)
}
if err := os.MkdirAll(path.Join(dataDir, "test-probe"), 0o755); err != nil {
    return fmt.Errorf("data-dir not writable: %w", err)
}

if err := runCompact(...); err != nil {
    if strings.Contains(err.Error(), "create meta fetcher") {
        // check dataDir permissions/space before retrying
    }
    return err
}

When it happens

Trigger: block.NewBaseFetcher returns non-nil because it cannot create/prepare conf.dataDir (permission denied, read-only filesystem, ENOSPC) or internal initialization fails.

Common situations: Data dir on a read-only volume or with wrong ownership; disk quota exceeded; dataDir pointing at an existing non-directory file.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• create syncer
• create working compact directory
• create working downsample directory
• get content of limit configuration
• create default tenant data dir

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/b624e00e4d26312a. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/compact.go:250

	duplicateBlocksFilter := block.NewDeduplicateFilter(conf.blockMetaFetchConcurrency)
	noCompactMarkerFilter := compact.NewGatherNoCompactionMarkFilter(logger, insBkt, conf.blockMetaFetchConcurrency)
	noDownsampleMarkerFilter := downsample.NewGatherNoDownsampleMarkFilter(logger, insBkt, conf.blockMetaFetchConcurrency)
	labelShardedMetaFilter := block.NewLabelShardedMetaFilter(relabelConfig, conf.dedupReplicaLabels...)
	consistencyDelayMetaFilter := block.NewConsistencyDelayMetaFilter(logger, conf.consistencyDelay, extprom.WrapRegistererWithPrefix("thanos_", reg))
	timePartitionMetaFilter := block.NewTimePartitionMetaFilter(conf.filterConf.MinTime, conf.filterConf.MaxTime)

	var blockLister block.Lister
	switch syncStrategy(conf.blockListStrategy) {
	case concurrentDiscovery:
		blockLister = block.NewConcurrentLister(logger, insBkt)
	case recursiveDiscovery:
		blockLister = block.NewRecursiveLister(logger, insBkt)
	default:
		return errors.Errorf("unknown sync strategy %s", conf.blockListStrategy)
	}
	baseMetaFetcher, err := block.NewBaseFetcher(logger, conf.blockMetaFetchConcurrency, insBkt, blockLister, conf.dataDir, extprom.WrapRegistererWithPrefix("thanos_", reg))
	if err != nil {
		return errors.Wrap(err, "create meta fetcher")
	}

	enableVerticalCompaction := conf.enableVerticalCompaction
	dedupReplicaLabels := strutil.ParseFlagLabels(conf.dedupReplicaLabels)
	if len(dedupReplicaLabels) > 0 {
		enableVerticalCompaction = true
		level.Info(logger).Log(
			"msg", "deduplication.replica-label specified, enabling vertical compaction", "dedupReplicaLabels", strings.Join(dedupReplicaLabels, ","),
		)
	}
	if enableVerticalCompaction {
		level.Info(logger).Log(
			"msg", "vertical compaction is enabled", "compact.enable-vertical-compaction", fmt.Sprintf("%v", conf.enableVerticalCompaction),
		)
	}
	var (
		api = blocksAPI.NewBlocksAPI(logger, conf.webConf.disableCORS, conf.label, flagsMap, insBkt)
		sy  *compact.Syncer

View on GitHub (pinned to 35b8b99117)