thanos-io/thanos · error

fetch Prometheus flags

Error message

fetch Prometheus flags

What it means

Inside validatePrometheus, the sidecar repeatedly calls client.ConfiguredFlags against the Prometheus /api/v1/status/flags endpoint. If the call fails with anything other than ErrFlagEndpointNotFound, the error is logged as a warning, retried, and wrapped as 'fetch Prometheus flags'; if the whole retry loop exits (ctx cancelled), the outer wrap also produces this message.

Solutions

1. Check Prometheus health and --prometheus.url reachability (curl <url>/api/v1/status/flags).
2. Wait for Prometheus to become fully started; the sidecar retries automatically every 2s.
3. If the outer wrap fired, the context was cancelled — investigate why the run group shut down.
4. Note ErrFlagEndpointNotFound is tolerated (older Prometheus); other failures are not.

resp, err := http.Get(promURL + "/api/v1/status/flags")
if err != nil || resp.StatusCode != 200 { /* defer validation */ }

if err != nil {
    var ctxErr bool = errors.Is(err, context.Canceled)
    if ctxErr { return err } // cancelled: abort, don't retry
}

When it happens

Trigger: Prometheus is unreachable (connection refused/timeout) or returns a failing HTTP status from /api/v1/status/flags while the sidecar validates its configuration; retries continue every 2s until success or ctx cancellation.

Common situations: Prometheus down during sidecar startup; wrong --prometheus.url; a load balancer stripping the status endpoint; ctx cancelled because the operator stopped Thanos during a long Prometheus outage.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• failed to validate prometheus flags
• failed to get prometheus version
• failed to get tsdb status from prometheus
• validate relabel config
• perform request against

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/3e6f0f39c7a5eca2. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/sidecar.go:505

		}, func(error) {
			cancel()
		})
	}

	level.Info(logger).Log("msg", "starting sidecar")
	return nil
}

func validatePrometheus(ctx context.Context, client *promclient.Client, logger log.Logger, conf *sidecarConfig, m *promMetadata) error {
	var (
		flagErr error
		flags   promclient.Flags
	)

	if err := runutil.Retry(2*time.Second, ctx.Done(), func() error {
		if flags, flagErr = client.ConfiguredFlags(ctx, m.promURL); flagErr != nil && flagErr != promclient.ErrFlagEndpointNotFound {
			level.Warn(logger).Log("msg", "failed to get Prometheus flags. Is Prometheus running? Retrying", "err", flagErr)
			return errors.Wrapf(flagErr, "fetch Prometheus flags")
		}
		return nil
	}); err != nil {
		return errors.Wrapf(err, "fetch Prometheus flags")
	}

	if flagErr != nil {
		level.Warn(logger).Log("msg", "failed to check Prometheus flags, due to potentially older Prometheus. No extra validation is done.", "err", flagErr)
		return nil
	}

	if flags.TSDBDelayCompact != "" {
		thanosMetaPath := filepath.Join(conf.tsdb.path, conf.shipper.metaFileName)
		if filepath.Clean(flags.TSDBDelayCompact) != filepath.Clean(thanosMetaPath) {
			return errors.Errorf(
				"found that Prometheus and Thanos use different paths for tracking block uploads. "+
					"Prometheus uses --storage.tsdb.delay-compact-file.path=%s while Thanos will write to %s, they must both use the same path.",
				flags.TSDBDelayCompact, thanosMetaPath,

View on GitHub (pinned to 35b8b99117)