{"record":{"id":"60b92750c6299700","repo":"dgraph-io/dgraph","slug":"failed-to-send-forward-request-v","errorCode":null,"errorMessage":"failed to send forward request: %v","messagePattern":"failed to send forward request: (.+?)","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"worker/import.go","lineNumber":387,"sourceCode":"\t\tglog.Errorf(\"[import] unable to connect to the leader of group [%v]\", groupId)\n\t\treturn fmt.Errorf(\"unable to connect to the leader of group [%v] : %v\", groupId, conn.ErrNoConnection)\n\t}\n\n\tcon := pl.Get()\n\tc := pb.NewWorkerClient(con)\n\talphaStream, err := c.StreamExtSnapshot(stream.Context())\n\tif err != nil {\n\t\tglog.Errorf(\"[import] failed to establish stream with leader: %v\", err)\n\t\treturn fmt.Errorf(\"failed to establish stream with leader: %v\", err)\n\t}\n\tglog.Infof(\"[import] [forward %d -> %d] start\", groups().Node.gid, groupId)\n\tglog.Infof(\"[import] [forward %v -> %d] start\", groups().Node.MyAddr, groups().Leader(groupId).Addr)\n\n\tglog.Infof(\"[import] sending forward true to leader of group [%v]\", groupId)\n\tforwardReq := &api.StreamExtSnapshotRequest{Forward: true}\n\tif err := alphaStream.Send(forwardReq); err != nil {\n\t\tglog.Errorf(\"[import] failed to send forward request: %v\", err)\n\t\treturn fmt.Errorf(\"failed to send forward request: %v\", err)\n\t}\n\n\treturn pipeTwoStream(stream, alphaStream, groupId)\n}\n\nfunc pipeTwoStream(in api.Dgraph_StreamExtSnapshotServer, out pb.Worker_StreamExtSnapshotClient, groupId uint32) error {\n\tcurrentGroup := groups().Node.gid\n\tctx := in.Context()\n\n\tfor {\n\t\tif err := ctx.Err(); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treq, err := in.Recv()\n\t\tif errors.Is(err, io.EOF) {\n\t\t\treturn nil\n\t\t}","sourceCodeStart":369,"sourceCodeEnd":405,"githubUrl":"https://github.com/dgraph-io/dgraph/blob/759e242be62c91f8d084da06ad0c8d21256d9c07/worker/import.go#L369-L405","documentation":"InStream must first tell the downstream leader that it is forwarding an external snapshot by sending api.StreamExtSnapshotRequest{Forward: true}. If alphaStream.Send fails, the handshake cannot complete and this error is returned with the underlying cause. It almost always means the gRPC stream to the leader died right after being opened.","triggerScenarios":"alphaStream.Send(forwardReq) returns err because the connection to the leader dropped, the leader cancelled the stream's context, flow-control backpressure timed out, or the server closed the stream immediately (e.g. it errored on its own startup).","commonSituations":"Leader crashed between RPC setup and first Send; network interruption mid-import startup; server-side import rejected the request and closed the stream before reading; gRPC max message/frame limits misconfigured.","solutions":["Read the wrapped err — io.EOF usually means the leader closed the stream; check the leader's logs for the server-side reason","Verify the leader process is still alive and re-establish connectivity, then retry the import","Ensure both sides use compatible StreamExtSnapshot message versions","Check for network devices between nodes that kill long-lived gRPC streams (idle timeouts)"],"exampleFix":"// before\nif err := alphaStream.Send(forwardReq); err != nil {\n    return fmt.Errorf(\"failed to send forward request: %v\", err)\n}\n// after: log the leader address alongside the cause for faster diagnosis\nif err := alphaStream.Send(forwardReq); err != nil {\n    glog.Errorf(\"[import] forward handshake to %s failed: %v\", pl.Addr, err)\n    return fmt.Errorf(\"failed to send forward request: %w\", err)\n}","handlingStrategy":"retry","validationCode":"// confirm leader stability immediately before import\nif err := pingLeader(leaderAddr); err != nil {\n    return fmt.Errorf(\"leader %s unhealthy before import: %w\", leaderAddr, err)\n}","typeGuard":null,"tryCatchPattern":"if err != nil && strings.Contains(err.Error(), \"failed to send forward request\") {\n    // stream died at handshake; check leader liveness, then retry with backoff\n    return retryImport(ctx, groupID)\n}","preventionTips":["Confirm the leader survives a few minutes before long imports","Watch leader logs during import startup","Disable aggressive connection idle timeouts on network gear","Keep both sides on compatible StreamExtSnapshot schemas"],"tags":["grpc","streaming","network","handshake"],"backgroundTag":"grpc-stream-send-failed","analyzedSha":"759e242be62c91f8d084da06ad0c8d21256d9c07","analyzedAt":"2026-09-01T14:42:12.034Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}