{"record":{"id":"aaf23cccd3fbf2e3","repo":"jaegertracing/jaeger","slug":"received-error-from-grpc-stream-w","errorCode":null,"errorMessage":"received error from grpc stream: %w","messagePattern":"received error from grpc stream: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/storage/v2/grpc/tracereader.go","lineNumber":98,"sourceCode":"\treturn func(yield func([]ptrace.Traces, error) bool) {\n\t\tquery := []*storage.GetTraceParams{}\n\t\tfor _, traceID := range traceIDs {\n\t\t\tquery = append(query, &storage.GetTraceParams{\n\t\t\t\tTraceId:   traceID.TraceID[:],\n\t\t\t\tStartTime: traceID.Start,\n\t\t\t\tEndTime:   traceID.End,\n\t\t\t})\n\t\t}\n\t\tstream, err := tr.client.GetTraces(ctx, &storage.GetTracesRequest{\n\t\t\tQuery: query,\n\t\t})\n\t\tif err != nil {\n\t\t\tyield(nil, fmt.Errorf(\"failed to execute GetTraces: %w\", err))\n\t\t\treturn\n\t\t}\n\t\tfor received, err := stream.Recv(); !errors.Is(err, io.EOF); received, err = stream.Recv() {\n\t\t\tif err != nil {\n\t\t\t\tyield(nil, fmt.Errorf(\"received error from grpc stream: %w\", err))\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif !yield([]ptrace.Traces{received.ToTraces()}, nil) {\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc (tr *TraceReader) GetServices(ctx context.Context) ([]string, error) {\n\tresp, err := tr.client.GetServices(ctx, &storage.GetServicesRequest{})\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to execute GetServices: %w\", err)\n\t}\n\treturn resp.Services, nil\n}\n\nfunc (tr *TraceReader) GetOperations(","sourceCodeStart":80,"sourceCodeEnd":116,"githubUrl":"https://github.com/jaegertracing/jaeger/blob/806f4447841ecdb60519f408b004a599d515f437/internal/storage/v2/grpc/tracereader.go#L80-L116","documentation":"While consuming the GetTraces server stream, stream.Recv() returned a non-EOF error mid-stream; the reader yields it wrapped as 'received error from grpc stream' and stops iteration. Traces already delivered remain valid.","triggerScenarios":"Server-side failure or connection reset while streaming a large trace set; deadline exceeded during a long stream; server crashes between messages.","commonSituations":"Very large GetTraces responses hitting timeouts or max message/stream limits; network interruption; remote storage OOM or restart mid-query.","solutions":["Inspect the wrapped gRPC status for the mid-stream cause","Reduce batch size / number of requested traces per call","Increase context deadline and max_recv_msg_size on both ends","Add retry logic that re-issues GetTraces for missing trace IDs"],"exampleFix":"// before\nfor traces, err := range reader.GetTraces(ctx, ids, q) {\n    if err != nil { return err }\n}\n// after\nfor traces, err := range reader.GetTraces(ctx, ids, q) {\n    if err != nil {\n        if status.Code(err) == codes.Unavailable { ids = missing(ids, got); continue }\n        return err\n    }\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"for traces, err := range reader.GetTraces(ctx, ids, q) {\n    if err != nil {\n        if status.Code(err) == codes.Unavailable {\n            // re-issue GetTraces for the not-yet-received IDs\n            continue\n        }\n        return err\n    }\n    got = append(got, traces...)\n}","preventionTips":["Limit traces per request to avoid long-lived streams","Increase server keepalive/max message settings for large traces","Monitor server for restarts/OOM during heavy queries"],"tags":["grpc","network","streaming"],"backgroundTag":"grpc-stream-error","analyzedSha":"806f4447841ecdb60519f408b004a599d515f437","analyzedAt":"2026-09-01T02:39:22.140Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T05:18:18.240Z"}