What causes "add config file %s to watcher" in thanos-io/thanos?

Verify the file exists and is readable: ls -l (or kubectl exec + ls in the container).

add config file to watcher

Error message

add config file %s to watcher

What it means

Thanos Reloader's Watch adds the main configuration file to an fsnotify watcher so config changes trigger a reload. This error is returned when watcher.addFile(r.cfgFile) fails — most often because the config file does not exist or is not readable at the given path — and the whole Watch (and the component startup using it) aborts.

Solutions

Verify the file exists and is readable: ls -l <path> (or kubectl exec + ls in the container).
Correct the flag/value pointing at the config file in the component's arguments.
Ensure the config file is actually mounted into the container (ConfigMap/volume) before startup.
Place configs on a local filesystem that supports inotify instead of some network mounts.

Example fix

// before
--config-file=/etc/thanos/wrong-name.yml
// after
--config-file=/etc/thanos/rules.yml

Defensive patterns

Strategy: validation

Validate before calling

info, err := os.Stat(cfgFile)
if err != nil || info.IsDir() {
    return fmt.Errorf("config file %s missing or not a file", cfgFile)
}

Try / catch

if err := reloader.Watch(ctx); err != nil {
    if strings.Contains(err.Error(), "add config file") {
        logger.Error("config file path invalid; check mount and flag value", "err", err)
        os.Exit(1)
    }
}

Prevention

Validate config paths at container startup with a pre-start check.
Mount configs before the process starts and verify with readiness probes.
Avoid network filesystems lacking inotify for watched configs.

When it happens

Trigger: Watch (pkg/reloader/reloader.go:289) is called with r.cfgFile set to a path that does not exist, is a directory instead of a file, lacks read permission, or is on an unwatchable filesystem; watcher.addFile returns the fsnotify/OS error, wrapped as 'add config file %s to watcher'.

Common situations: Typo in the --rule-dir/--config-file flag path; container image where the config was never mounted; file on a tmpfs/network mount unsupported by inotify; permissions changed after deployment.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/bf746bf1dd0cec2a. Report an issue: GitHub.

Appendix: source

Thrown at pkg/reloader/reloader.go:289

// watch interval.
func (r *Reloader) Watch(ctx context.Context) error {
	if r.cfgFile == "" && len(r.cfgDirs) == 0 && len(r.watchedDirs) == 0 {
		level.Info(r.logger).Log("msg", "nothing to be watched")
		<-ctx.Done()
		return nil
	}

	if _, ok := r.tr.(*PIDReloader); ok {
		level.Info(r.logger).Log("msg", "reloading via process signal")
	} else {
		level.Info(r.logger).Log("msg", "reloading via HTTP")
	}

	defer runutil.CloseWithLogOnErr(r.logger, r.watcher, "config watcher close")

	if r.cfgFile != "" {
		if err := r.watcher.addFile(r.cfgFile); err != nil {
			return errors.Wrapf(err, "add config file %s to watcher", r.cfgFile)
		}
		initialSyncCtx, initialSyncCancel := context.WithTimeout(ctx, r.watchInterval)
		err := r.apply(initialSyncCtx)
		initialSyncCancel()
		if err != nil {
			return err
		}
	}

	for _, cfgDir := range r.cfgDirs {
		dir := cfgDir.Dir
		if err := r.watcher.addDirectory(dir); err != nil {
			return errors.Wrapf(err, "add directory %s to watcher", dir)
		}
	}

	if r.watchInterval == 0 {
		// Skip watching the file-system.

View on GitHub (pinned to 35b8b99117)