jaegertracing/jaeger · error

failed to execute GetTraces

Error message

failed to execute GetTraces: %w

What it means

Inside the GetTraces iterator, the initial gRPC call to open the GetTraces stream failed; the error is yielded to the push iterator wrapped as 'failed to execute GetTraces'. No traces are delivered and the iteration terminates.

Solutions

1. Inspect the wrapped gRPC status to classify the failure
2. Retry with backoff for codes.Unavailable/DeadlineExceeded
3. Verify remote storage server health and endpoint config
4. Ensure the context deadline is adequate for trace-fetch workloads

Example fix

// before
for traces, err := range reader.GetTraces(ctx, ids, q) { ... }
// after
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
for traces, err := range reader.GetTraces(ctx, ids, q) {
    if err != nil && status.Code(err) == codes.Unavailable { /* retry */ }
}

// validate trace IDs before querying
for _, id := range traceIDs {
    if _, err := pcommon.TraceIDFromHex(id); err != nil {
        return fmt.Errorf("invalid trace ID %q", id)
    }
}

for traces, err := range reader.GetTraces(ctx, ids, q) {
    if err != nil {
        if status.Code(err) == codes.Unavailable || status.Code(err) == codes.DeadlineExceeded {
            // retry with backoff
        }
        return err
    }
}

When it happens

Trigger: Iterating TraceReader.GetTraces(ctx, traceIDs, query) when the RPC setup fails: server unavailable, deadline exceeded, request rejected (auth/tenancy), or invalid query parameters.

Common situations: Remote storage down at query time; context deadline too short for large queries; invalid trace IDs format; tenancy headers missing.

Related errors

• error creating reader client connection
• error creating writer client connection
• failed to listen on gRPC port
• invalid gRPC server host:port
• query server failed to initialize listener

AI-assisted analysis of jaegertracing/jaeger@806f444784 (2026-09-01). Data as JSON: /api/errors/bcbe65be774a5b12. Report an issue: GitHub.

Appendix: source

Thrown at internal/storage/v2/grpc/tracereader.go:93

func (tr *TraceReader) GetTraces(
	ctx context.Context,
	traceIDs ...tracestore.GetTraceParams,
) iter.Seq2[[]ptrace.Traces, error] {
	return func(yield func([]ptrace.Traces, error) bool) {
		query := []*storage.GetTraceParams{}
		for _, traceID := range traceIDs {
			query = append(query, &storage.GetTraceParams{
				TraceId:   traceID.TraceID[:],
				StartTime: traceID.Start,
				EndTime:   traceID.End,
			})
		}
		stream, err := tr.client.GetTraces(ctx, &storage.GetTracesRequest{
			Query: query,
		})
		if err != nil {
			yield(nil, fmt.Errorf("failed to execute GetTraces: %w", err))
			return
		}
		for received, err := stream.Recv(); !errors.Is(err, io.EOF); received, err = stream.Recv() {
			if err != nil {
				yield(nil, fmt.Errorf("received error from grpc stream: %w", err))
				return
			}
			if !yield([]ptrace.Traces{received.ToTraces()}, nil) {
				return
			}
		}
	}
}

func (tr *TraceReader) GetServices(ctx context.Context) ([]string, error) {
	resp, err := tr.client.GetServices(ctx, &storage.GetServicesRequest{})
	if err != nil {
		return nil, fmt.Errorf("failed to execute GetServices: %w", err)

View on GitHub (pinned to 806f444784)