thanos-io/thanos · error

no query API server reachable

Error message

no query API server reachable

What it means

The rule component builds a query client by iterating configured query API endpoints (--query / --query-url) and returns errors.Errorf("no query API server reachable") when none of the configured endpoints succeeded the clientCreation check. Without a reachable Querier the rule evaluator cannot run queries, so startup of the query-pool fails.

Solutions

1. Verify each --query address with curl (http://<host>:<port>/-/ready) from the rule pod.
2. Correct DNS/service names and ports in --query / --query-url flags.
3. Ensure Querier replicas are running and reachable (network policies, service selectors).
4. Provide multiple query endpoints or rely on discovery groups so one failure doesn't kill the pool; restart Rule after Querier is up.

Example fix

// before
thanos rule --query=querier.monitoring.svc:9999        # wrong port
// after
thanos rule --query=querier.monitoring.svc:9090

# from the rule pod, before start:
for q in $QUERY_ENDPOINTS; do curl -sf "http://$q/-/ready" || exit 1; done

// re-create the query pool with backoff
err := buildQueryPool(...)
for errors.Cause(err).Error() == "no query API server reachable" && attempts < max {
  time.Sleep(backoff); err = buildQueryPool(...)
}

When it happens

Trigger: All --query addresses are unreachable, DNS-unresolvable, or reject connections at rule startup; the query address list ends up empty after discovery filtering.

Common situations: Querier pod not yet up when Rule starts (ordering in the same manifest); wrong --query host/port; network policy or service name typo in Kubernetes; Querier replicas all down.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• proxy Series()
• Not ready
• setup gRPC server
• Not ready
• error starting web server

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/90acf8cbbe51a673. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/rule.go:1053

					if err != nil {
						level.Error(logger).Log("err", err, "query", qs)
						continue
					}

					warnings := make([]string, 0, len(result.Warnings))
					for _, warn := range result.Warnings {
						warnings = append(warnings, warn.Error())
					}
					warnings = filterOutPromQLWarnings(warnings, logger, qs)
					if len(warnings) > 0 {
						ruleEvalWarnings.WithLabelValues(strings.ToLower(partialResponseStrategy.String())).Inc()
						level.Warn(logger).Log("warnings", strings.Join(warnings, ", "), "query", qs)
					}

					return v, nil
				}
			}
			return nil, errors.Errorf("no query API server reachable")
		}
	}
}

func addDiscoveryGroups(g *run.Group, c *clientconfig.HTTPClient, interval time.Duration, logger log.Logger) {
	ctx, cancel := context.WithCancel(context.Background())
	g.Add(func() error {
		c.Discover(ctx)
		return nil
	}, func(error) {
		cancel()
	})

	g.Add(func() error {
		runutil.RepeatInfinitely(logger, interval, ctx.Done(), func() error {
			return c.Resolve(ctx)
		})
		return nil

View on GitHub (pinned to 35b8b99117)