jaegertracing/jaeger · error

received error from grpc stream: %w

Error message

received error from grpc stream: %w

What it means

While consuming the GetTraces server stream, stream.Recv() returned a non-EOF error mid-stream; the reader yields it wrapped as 'received error from grpc stream' and stops iteration. Traces already delivered remain valid.

Source

Thrown at internal/storage/v2/grpc/tracereader.go:98

	return func(yield func([]ptrace.Traces, error) bool) {
		query := []*storage.GetTraceParams{}
		for _, traceID := range traceIDs {
			query = append(query, &storage.GetTraceParams{
				TraceId:   traceID.TraceID[:],
				StartTime: traceID.Start,
				EndTime:   traceID.End,
			})
		}
		stream, err := tr.client.GetTraces(ctx, &storage.GetTracesRequest{
			Query: query,
		})
		if err != nil {
			yield(nil, fmt.Errorf("failed to execute GetTraces: %w", err))
			return
		}
		for received, err := stream.Recv(); !errors.Is(err, io.EOF); received, err = stream.Recv() {
			if err != nil {
				yield(nil, fmt.Errorf("received error from grpc stream: %w", err))
				return
			}
			if !yield([]ptrace.Traces{received.ToTraces()}, nil) {
				return
			}
		}
	}
}

func (tr *TraceReader) GetServices(ctx context.Context) ([]string, error) {
	resp, err := tr.client.GetServices(ctx, &storage.GetServicesRequest{})
	if err != nil {
		return nil, fmt.Errorf("failed to execute GetServices: %w", err)
	}
	return resp.Services, nil
}

func (tr *TraceReader) GetOperations(

View on GitHub (pinned to 806f444784)

Solutions

1. Inspect the wrapped gRPC status for the mid-stream cause
2. Reduce batch size / number of requested traces per call
3. Increase context deadline and max_recv_msg_size on both ends
4. Add retry logic that re-issues GetTraces for missing trace IDs

Example fix

// before
for traces, err := range reader.GetTraces(ctx, ids, q) {
    if err != nil { return err }
}
// after
for traces, err := range reader.GetTraces(ctx, ids, q) {
    if err != nil {
        if status.Code(err) == codes.Unavailable { ids = missing(ids, got); continue }
        return err
    }
}

for traces, err := range reader.GetTraces(ctx, ids, q) {
    if err != nil {
        if status.Code(err) == codes.Unavailable {
            // re-issue GetTraces for the not-yet-received IDs
            continue
        }
        return err
    }
    got = append(got, traces...)
}

When it happens

Trigger: Server-side failure or connection reset while streaming a large trace set; deadline exceeded during a long stream; server crashes between messages.

Common situations: Very large GetTraces responses hitting timeouts or max message/stream limits; network interruption; remote storage OOM or restart mid-query.

Related errors

• invalid gRPC server host:port: %w
• query server failed to initialize listener: %w
• failed to listen on gRPC port: %w
• failed to read span rows: %w
• failed to read trace ID rows: %w

AI-assisted analysis of jaegertracing/jaeger@806f444784 (2026-09-01). Data as JSON: /api/errors/aaf23cccd3fbf2e3. Report an issue: GitHub.