{"record":{"id":"24644737829e744d","repo":"jaegertracing/jaeger","slug":"failed-getting-error-metrics-w","errorCode":null,"errorMessage":"failed getting error metrics: %w","messagePattern":"failed getting error metrics: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/storage/metricstore/prometheus/metricstore/reader.go","lineNumber":221,"sourceCode":"\n// GetErrorRates gets the error rate metrics for the given set of error rate query parameters.\nfunc (m MetricsReader) GetErrorRates(ctx context.Context, requestParams *metricstore.ErrorRateQueryParameters) (*metrics.MetricFamily, error) {\n\tmetricsParams := metricsQueryParams{\n\t\tBaseQueryParameters: requestParams.BaseQueryParameters,\n\t\tmetricName:          \"service_error_rate\",\n\t\tmetricDesc:          \"error rate, computed as a fraction of errors/sec over calls/sec, grouped by service\",\n\t\tbuildPromQuery: func(p promQueryParams) string {\n\t\t\treturn fmt.Sprintf(\n\t\t\t\t// Note: p.spanKindFilter can be \"\"; trailing commas are okay within a timeseries selection.\n\t\t\t\t`sum(rate(%s{service_name =~ %q, status_code = \"STATUS_CODE_ERROR\", %s}[%s])) by (%s) / sum(rate(%s{service_name =~ %q, %s}[%s])) by (%s)`,\n\t\t\t\tm.callsMetricName, p.serviceFilter, p.spanKindFilter, p.rate, p.groupBy,\n\t\t\t\tm.callsMetricName, p.serviceFilter, p.spanKindFilter, p.rate, p.groupBy,\n\t\t\t)\n\t\t},\n\t}\n\terrorMetrics, err := m.executeQuery(ctx, metricsParams)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed getting error metrics: %w\", err)\n\t}\n\t// Non-zero error rates are available.\n\tif len(errorMetrics.Metrics) > 0 {\n\t\treturn errorMetrics, nil\n\t}\n\n\t// Check for the presence of call rate metrics to differentiate the absence of error rate from\n\t// the absence of call rate metrics altogether.\n\tcallMetrics, err := m.GetCallRates(ctx, &metricstore.CallRateQueryParameters{BaseQueryParameters: requestParams.BaseQueryParameters})\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed getting call metrics: %w\", err)\n\t}\n\t// No call rate metrics are available, and by association, means no error rate metrics are available.\n\tif len(callMetrics.Metrics) == 0 {\n\t\treturn errorMetrics, nil\n\t}\n\n\t// Non-zero call rate metrics are available, which implies that there are just no errors, so we report a zero error rate.","sourceCodeStart":203,"sourceCodeEnd":239,"githubUrl":"https://github.com/jaegertracing/jaeger/blob/806f4447841ecdb60519f408b004a599d515f437/internal/storage/metricstore/prometheus/metricstore/reader.go#L203-L239","documentation":"GetErrorRates in the Prometheus metricstore reader wraps a failure from executeQuery (the underlying Prometheus range query for error-rate metrics, e.g. calls{error=true} / calls) with this prefix. It means the Prometheus range query itself failed — network issue, bad query, or Prometheus server error — not that error metrics are absent. The original Prometheus client error is preserved via %w.","triggerScenarios":"Calling GetErrorRates when the internal QueryRange call via m.client.QueryRange fails: unreachable Prometheus endpoint, invalid query built from the query parameters, HTTP error from the Prometheus API, or context cancellation/timeout.","commonSituations":"Misconfigured --prometheus.server-url, Prometheus down or overloaded, network/proxy issues between Jaeger and Prometheus, or query parameters (service, spanKind, groupBy) producing a query Prometheus rejects.","solutions":["Check Prometheus connectivity and the configured server URL (curl the /api/v1 endpoint from the Jaeger host).","Inspect the wrapped (%w) inner error and Prometheus server logs for the actual cause.","Verify the query parameters: the service and span-kind filters must match data actually present in Prometheus.","Increase the client timeout / check ctx deadline if queries time out on large ranges."],"exampleFix":"// before: opaque failure\nerrorMetrics, err := m.executeQuery(ctx, metricsParams)\n// after: log/inspect the wrapped cause\nif err != nil {\n    logger.Error(\"error rates query failed\", zap.Error(err)) // err already wraps the Prometheus client error\n    return nil, fmt.Errorf(\"failed getting error metrics: %w\", err)\n}","handlingStrategy":"try-catch","validationCode":"resp, err := http.Get(promURL + \"/-/ready\")\nif err != nil || resp.StatusCode != 200 {\n    return fmt.Errorf(\"prometheus at %s is not reachable\", promURL)\n}","typeGuard":null,"tryCatchPattern":"if err != nil {\n    var ae *promapi.Error\n    if errors.As(err, &ae) { /* inspect Prometheus API error */ }\n    return nil, fmt.Errorf(\"failed getting error metrics: %w\", err)\n}","preventionTips":["Health-check the Prometheus endpoint before serving metrics APIs.","Set sensible client timeouts and propagate request contexts.","Monitor Jaeger-to-Prometheus connectivity in dashboards/alerts."],"tags":["prometheus","metrics","storage","network"],"backgroundTag":"prometheus-query-failed","analyzedSha":"806f4447841ecdb60519f408b004a599d515f437","analyzedAt":"2026-09-01T02:39:22.140Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T05:18:18.240Z"}