{"record":{"id":"8f0a7ad043fdfed6","repo":"dgraph-io/dgraph","slug":"recv-final-downstream-d-w","errorCode":null,"errorMessage":"recv final downstream(%d): %w","messagePattern":"recv final downstream\\((.+?)\\): %w","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"worker/import.go","lineNumber":430,"sourceCode":"\n\t\tif req.Pkt.Done {\n\t\t\t// Forward Done, half-close downstream send.\n\t\t\tif err := out.Send(&api.StreamExtSnapshotRequest{Pkt: req.Pkt}); err != nil && !errors.Is(err, io.EOF) {\n\t\t\t\treturn fmt.Errorf(\"send done downstream(%d): %w\", groupId, err)\n\t\t\t}\n\t\t\t_ = out.CloseSend()\n\n\t\t\t// Drain downstream and relay upstream until Finish=true.\n\t\t\tfor {\n\t\t\t\tif err := ctx.Err(); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tresp, err := out.Recv()\n\t\t\t\tif errors.Is(err, io.EOF) {\n\t\t\t\t\treturn fmt.Errorf(\"downstream(%d) closed before Finish=true\", groupId)\n\t\t\t\t}\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"recv final downstream(%d): %w\", groupId, err)\n\t\t\t\t}\n\t\t\t\tif err := in.Send(resp); err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"relay final upstream: %w\", err)\n\t\t\t\t}\n\t\t\t\tif resp.Finish {\n\t\t\t\t\tglog.Infof(\"[import] [forward %d -> %d] finish\", currentGroup, groupId)\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// Normal data chunk: send -> wait ack -> send upstream ack.\n\t\tif err := out.Send(&api.StreamExtSnapshotRequest{Pkt: req.Pkt}); err != nil {\n\t\t\treturn fmt.Errorf(\"send data downstream(%d): %w\", groupId, err)\n\t\t}\n\t\tif _, err := out.Recv(); err != nil {\n\t\t\treturn fmt.Errorf(\"ack data downstream(%d): %w\", groupId, err)\n\t\t}","sourceCodeStart":412,"sourceCodeEnd":448,"githubUrl":"https://github.com/dgraph-io/dgraph/blob/759e242be62c91f8d084da06ad0c8d21256d9c07/worker/import.go#L412-L448","documentation":"While draining the downstream for the Finish frame, out.Recv() failed with a non-EOF error. The relay cannot learn the import outcome and aborts with this wrapped error identifying the group. Unlike EOF (816), this indicates an active transport/RPC failure rather than a clean close.","triggerScenarios":"out.Recv() returns codes.Canceled (context cancelled), codes.Unavailable (connection lost), DeadlineExceeded, or a transport error while waiting for the leader's final frames.","commonSituations":"Network partition during the final phase; leader restarted; overall import context deadline hit during a slow final flush; keepalive probe failure killing the connection.","solutions":["Inspect the wrapped status code: Unavailable/transport => retry the import; Canceled => find what cancelled the context","Raise context deadlines to cover slow finalization on large snapshots","Verify leader stability and connectivity, then re-run the import","Enable gRPC keepalives to detect and survive idle periods gracefully"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"ctx, cancel := context.WithTimeout(context.Background(), generousDeadlineForSnapshot)\ndefer cancel()","typeGuard":"func isTransientRecvFailure(err error) bool {\n    c := status.Code(err)\n    return c == codes.Unavailable || c == codes.DeadlineExceeded\n}","tryCatchPattern":"if err != nil && strings.Contains(err.Error(), \"recv final downstream\") {\n    switch status.Code(errors.Unwrap(err)) {\n    case codes.Unavailable, codes.DeadlineExceeded: /* retry import */\n    case codes.Canceled: /* find the cancelling caller */\n    }\n}","preventionTips":["Size context deadlines to the snapshot size, not a fixed few minutes","Enable gRPC keepalives to hold idle streams","Verify network stability between alphas before large imports","Retry with backoff on Unavailable/DeadlineExceeded"],"tags":["grpc","streaming","network","timeout"],"backgroundTag":"downstream-stream-recv-failed","analyzedSha":"759e242be62c91f8d084da06ad0c8d21256d9c07","analyzedAt":"2026-09-01T14:42:12.034Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}