jaegertracing/jaeger · error

failed executing metrics query: %w

Error message

failed executing metrics query: %w

What it means

executeQuery runs a Prometheus RangeQuery (client.QueryRange) for latencies, call rates, or error rates. When the query execution fails, the error is wrapped with this prefix, logged to the active tracing span, and returned. It is the single choke point for all Prometheus metric-store query failures in this reader.

Source

Thrown at internal/storage/metricstore/prometheus/metricstore/reader.go:272

func (m MetricsReader) executeQuery(ctx context.Context, p metricsQueryParams) (*metrics.MetricFamily, error) {
	if p.GroupByOperation {
		p.metricName = strings.Replace(p.metricName, "service", "service_operation", 1)
		p.metricDesc += " & operation"
	}
	promQuery := m.buildPromQuery(p)

	ctx, span := startSpanForQuery(ctx, p.metricName, promQuery, m.tracer)
	defer span.End()

	queryRange := promapi.Range{
		Start: p.EndTime.Add(-1 * *p.Lookback),
		End:   *p.EndTime,
		Step:  *p.Step,
	}

	mv, warnings, err := m.client.QueryRange(ctx, promQuery, queryRange)
	if err != nil {
		err = fmt.Errorf("failed executing metrics query: %w", err)
		logErrorToSpan(span, err)
		return &metrics.MetricFamily{}, err
	}
	if len(warnings) > 0 {
		m.logger.Warn("Warnings detected on Prometheus query", zap.Any("warnings", warnings), zap.String("query", promQuery), zap.Any("range", queryRange))
	}

	m.logger.Debug("Prometheus query results", zap.String("results", mv.String()), zap.String("query", promQuery), zap.Any("range", queryRange))

	return m.metricsTranslator.ToDomainMetricsFamily(
		p.metricName,
		p.metricDesc,
		mv,
	)
}

func (m MetricsReader) buildPromQuery(metricsParams metricsQueryParams) string {
	groupBy := []string{"service_name"}

View on GitHub (pinned to 806f444784)

Solutions

1. Inspect the wrapped error for the exact Prometheus client cause (URL, status code, query).
2. Verify --prometheus.server-url, TLS, and auth settings against your Prometheus deployment.
3. Reduce the query range or increase the step to avoid timeouts on heavy queries.
4. Test the generated promql manually against Prometheus (it is logged on warnings and available via the span).

Example fix

// before: retrying blindly
resp, err := getMetrics()
// after: fail fast and surface the underlying cause
if err != nil {
    var apiErr *promapi.Error
    if errors.As(err, &apiErr) {
        logger.Error("prometheus rejected query", zap.String("query", promQuery), zap.Error(apiErr))
    }
    return nil, err
}

if resp, err := http.Get(promURL + "/api/v1/status/buildinfo"); err != nil || resp.StatusCode != 200 {
    return errors.New("prometheus API unavailable")
}

mv, warnings, err := m.client.QueryRange(ctx, promQuery, queryRange)
if err != nil {
    if ctx.Err() != nil { return ctx.Err() } // don't retry cancelled contexts
    err = fmt.Errorf("failed executing metrics query: %w", err)
    logErrorToSpan(span, err)
    return &metrics.MetricFamily{}, err
}

When it happens

Trigger: Any of GetLatencies, GetCallRates, or GetErrorRates reaching executeQuery where m.client.QueryRange returns an error: HTTP failure, non-2xx from the Prometheus API, malformed promql, or context timeout.

Common situations: Prometheus unreachable (wrong URL/DNS), TLS misconfiguration, Prometheus 422/400 for invalid queries, timeouts on large time ranges with small steps.

Related errors

• failed executing metrics query: %w
• failed getting error metrics: %w
• failed getting call metrics: %w
• unexpected metrics ValueType: %s
• failed to query trace: %w

AI-assisted analysis of jaegertracing/jaeger@806f444784 (2026-09-01). Data as JSON: /api/errors/415e87145e910ce1. Report an issue: GitHub.