jaegertracing/jaeger · error

failed getting error metrics: %w

Error message

failed getting error metrics: %w

What it means

GetErrorRates in the Prometheus metricstore reader wraps a failure from executeQuery (the underlying Prometheus range query for error-rate metrics, e.g. calls{error=true} / calls) with this prefix. It means the Prometheus range query itself failed — network issue, bad query, or Prometheus server error — not that error metrics are absent. The original Prometheus client error is preserved via %w.

Source

Thrown at internal/storage/metricstore/prometheus/metricstore/reader.go:221

// GetErrorRates gets the error rate metrics for the given set of error rate query parameters.
func (m MetricsReader) GetErrorRates(ctx context.Context, requestParams *metricstore.ErrorRateQueryParameters) (*metrics.MetricFamily, error) {
	metricsParams := metricsQueryParams{
		BaseQueryParameters: requestParams.BaseQueryParameters,
		metricName:          "service_error_rate",
		metricDesc:          "error rate, computed as a fraction of errors/sec over calls/sec, grouped by service",
		buildPromQuery: func(p promQueryParams) string {
			return fmt.Sprintf(
				// Note: p.spanKindFilter can be ""; trailing commas are okay within a timeseries selection.
				`sum(rate(%s{service_name =~ %q, status_code = "STATUS_CODE_ERROR", %s}[%s])) by (%s) / sum(rate(%s{service_name =~ %q, %s}[%s])) by (%s)`,
				m.callsMetricName, p.serviceFilter, p.spanKindFilter, p.rate, p.groupBy,
				m.callsMetricName, p.serviceFilter, p.spanKindFilter, p.rate, p.groupBy,
			)
		},
	}
	errorMetrics, err := m.executeQuery(ctx, metricsParams)
	if err != nil {
		return nil, fmt.Errorf("failed getting error metrics: %w", err)
	}
	// Non-zero error rates are available.
	if len(errorMetrics.Metrics) > 0 {
		return errorMetrics, nil
	}

	// Check for the presence of call rate metrics to differentiate the absence of error rate from
	// the absence of call rate metrics altogether.
	callMetrics, err := m.GetCallRates(ctx, &metricstore.CallRateQueryParameters{BaseQueryParameters: requestParams.BaseQueryParameters})
	if err != nil {
		return nil, fmt.Errorf("failed getting call metrics: %w", err)
	}
	// No call rate metrics are available, and by association, means no error rate metrics are available.
	if len(callMetrics.Metrics) == 0 {
		return errorMetrics, nil
	}

	// Non-zero call rate metrics are available, which implies that there are just no errors, so we report a zero error rate.

View on GitHub (pinned to 806f444784)

Solutions

1. Check Prometheus connectivity and the configured server URL (curl the /api/v1 endpoint from the Jaeger host).
2. Inspect the wrapped (%w) inner error and Prometheus server logs for the actual cause.
3. Verify the query parameters: the service and span-kind filters must match data actually present in Prometheus.
4. Increase the client timeout / check ctx deadline if queries time out on large ranges.

Example fix

// before: opaque failure
errorMetrics, err := m.executeQuery(ctx, metricsParams)
// after: log/inspect the wrapped cause
if err != nil {
    logger.Error("error rates query failed", zap.Error(err)) // err already wraps the Prometheus client error
    return nil, fmt.Errorf("failed getting error metrics: %w", err)
}

resp, err := http.Get(promURL + "/-/ready")
if err != nil || resp.StatusCode != 200 {
    return fmt.Errorf("prometheus at %s is not reachable", promURL)
}

if err != nil {
    var ae *promapi.Error
    if errors.As(err, &ae) { /* inspect Prometheus API error */ }
    return nil, fmt.Errorf("failed getting error metrics: %w", err)
}

When it happens

Trigger: Calling GetErrorRates when the internal QueryRange call via m.client.QueryRange fails: unreachable Prometheus endpoint, invalid query built from the query parameters, HTTP error from the Prometheus API, or context cancellation/timeout.

Common situations: Misconfigured --prometheus.server-url, Prometheus down or overloaded, network/proxy issues between Jaeger and Prometheus, or query parameters (service, spanKind, groupBy) producing a query Prometheus rejects.

Related errors

• failed getting call metrics: %w
• failed executing metrics query: %w
• cannot find metrics storage factory: %w
• failed executing metrics query: %w
• unexpected metrics ValueType: %s

AI-assisted analysis of jaegertracing/jaeger@806f444784 (2026-09-01). Data as JSON: /api/errors/24644737829e744d. Report an issue: GitHub.