thanos-io/thanos · error

adding path to file watcher

Error message

adding path %s to file watcher

What it means

NewConfigWatcher wraps a failure from watcher.Add(path) when the OS refuses to watch the given hashring config path. fsnotify can only watch files that exist and are accessible to the process, so this error means the configured path is missing, is not a regular file, or is not readable.

Solutions

1. Verify the path exists and is readable by the Receive process user before startup (ls -l / test -r)
2. Fix the --hashrings-file flag value or the volume mount so the file is present at startup
3. Add an init container or entrypoint wait-loop that blocks until the config file exists
4. Check mount permissions (readOnly ConfigMap mounts owned by root vs non-root process)

Example fix

// before (file may not exist)
watcher, _ := receive.NewConfigWatcher(logger, cfgPath, interval)
// after
for {
	if _, err := os.Stat(cfgPath); err == nil { break }
	time.Sleep(time.Second)
}
watcher, err := receive.NewConfigWatcher(logger, cfgPath, interval)

info, err := os.Stat(path)
if err != nil { return fmt.Errorf("hashring file missing: %w", err) }
if info.IsDir() { return errors.New("path is a directory, not a file") }
f, err := os.Open(path)
if err != nil { return fmt.Errorf("not readable: %w", err) }
f.Close()

w, err := receive.NewConfigWatcher(logger, path, interval)
if err != nil && strings.Contains(err.Error(), "adding path") {
	// wait for the file to appear / fix mount, then retry
}

When it happens

Trigger: NewConfigWatcher called with a path that does not exist yet, points to a directory without permission, or was deleted between existence check and Add (Kubernetes symlink swap races).

Common situations: Wrong --hashrings-file path or typo; ConfigMap volume not mounted before startup; file created by a sidecar after Receive starts; permission mismatch between the mount and the running user.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• level is bigger then default set of
• unknown sync strategy
• create syncer
• get compaction levels
• penalty based deduplication needs at least one replica…

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/b4c63f1174f0b9f8. Report an issue: GitHub.

Appendix: source

Thrown at pkg/receive/config.go:193

	hashringNodesGauge   *prometheus.GaugeVec
	hashringTenantsGauge *prometheus.GaugeVec

	// lastLoadedConfigHash is the hash of the last successfully loaded configuration.
	lastLoadedConfigHash float64
}

// NewConfigWatcher creates a new ConfigWatcher.
func NewConfigWatcher(logger log.Logger, reg prometheus.Registerer, path string, interval model.Duration) (*ConfigWatcher, error) {
	if logger == nil {
		logger = log.NewNopLogger()
	}

	watcher, err := fsnotify.NewWatcher()
	if err != nil {
		return nil, errors.Wrap(err, "creating file watcher")
	}
	if err := watcher.Add(path); err != nil {
		return nil, errors.Wrapf(err, "adding path %s to file watcher", path)
	}

	c := &ConfigWatcher{
		ch:       make(chan []HashringConfig),
		path:     path,
		interval: time.Duration(interval),
		logger:   logger,
		watcher:  watcher,
		hashGauge: promauto.With(reg).NewGauge(
			prometheus.GaugeOpts{
				Name: "thanos_receive_config_hash",
				Help: "Hash of the currently loaded hashring configuration file.",
			}),
		successGauge: promauto.With(reg).NewGauge(
			prometheus.GaugeOpts{
				Name: "thanos_receive_config_last_reload_successful",
				Help: "Whether the last hashring configuration file reload attempt was successful.",
			}),

View on GitHub (pinned to 35b8b99117)