{"record":{"id":"00665b0cf059c136","repo":"apache/beam","slug":"stagefile-chunk-send-failed","errorCode":null,"errorMessage":"StageFile chunk send failed","messagePattern":"StageFile chunk send failed","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/go/pkg/beam/runners/universal/runnerlib/stage.go","lineNumber":165,"sourceCode":"\t}\n\tdefer fd.Close()\n\n\tdata := make([]byte, 1<<20)\n\tfor {\n\t\tn, err := fd.Read(data)\n\t\tif n > 0 {\n\t\t\tsendErr := stream.Send(&jobpb.ArtifactResponseWrapper{\n\t\t\t\tResponse: &jobpb.ArtifactResponseWrapper_GetArtifactResponse{\n\t\t\t\t\tGetArtifactResponse: &jobpb.GetArtifactResponse{\n\t\t\t\t\t\tData: data[:n],\n\t\t\t\t\t},\n\t\t\t\t}})\n\t\t\tif sendErr == io.EOF {\n\t\t\t\treturn sendErr\n\t\t\t}\n\n\t\t\tif sendErr != nil {\n\t\t\t\treturn errors.Wrap(sendErr, \"StageFile chunk send failed\")\n\t\t\t}\n\t\t}\n\n\t\tif err == io.EOF {\n\t\t\tsendErr := stream.Send(&jobpb.ArtifactResponseWrapper{\n\t\t\t\tIsLast: true,\n\t\t\t\tResponse: &jobpb.ArtifactResponseWrapper_GetArtifactResponse{\n\t\t\t\t\tGetArtifactResponse: &jobpb.GetArtifactResponse{},\n\t\t\t\t}})\n\t\t\treturn sendErr\n\t\t}\n\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n}\n","sourceCodeStart":147,"sourceCodeEnd":183,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/runners/universal/runnerlib/stage.go#L147-L183","documentation":"While streaming 1MB chunks of a staged artifact to the job server, stream.Send returned an error other than io.EOF, wrapped as 'StageFile chunk send failed'. This means the gRPC reverse artifact retrieval stream broke mid-upload.","triggerScenarios":"The gRPC connection to the job server drops, times out, or the server cancels the RPC while sending artifact chunks; any non-EOF send error during the upload loop.","commonSituations":"Uploading large artifacts over an unstable network; job server restarted or deadline exceeded during staging; proxy or firewall killing long-lived gRPC streams.","solutions":["Retry the pipeline launch; these failures are often transient network issues.","Verify connectivity to the job server endpoint (--endpoint) and that it is running and healthy.","Check for proxy/gRPC payload or timeout limits and increase deadlines if needed.","Inspect job server logs for stream cancellation or resource limits."],"exampleFix":"// before\nif sendErr != nil {\n\treturn errors.Wrap(sendErr, \"StageFile chunk send failed\")\n}\n// after\nif sendErr != nil {\n\tif status.Code(sendErr) == codes.Unavailable {\n\t\treturn retryable(sendErr) // caller retries with backoff\n\t}\n\treturn errors.Wrap(sendErr, \"StageFile chunk send failed\")\n}","handlingStrategy":"retry","validationCode":"conn, err := grpc.Dial(endpoint, grpc.WithBlock(), grpc.WithTimeout(5*time.Second))\nif err != nil {\n\treturn fmt.Errorf(\"job server %s unreachable: %w\", endpoint, err)\n}","typeGuard":null,"tryCatchPattern":"err := stageFiles(...)\nif err != nil {\n\tif status.Code(errors.Cause(err)) == codes.Unavailable || status.Code(errors.Cause(err)) == codes.DeadlineExceeded {\n\t\t// retry staging with exponential backoff\n\t}\n}","preventionTips":["Check job server health before launching.","Avoid staging very large artifacts over unstable networks.","Set generous gRPC deadlines for artifact staging."],"tags":["go","beam","grpc","network"],"backgroundTag":"network-request-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}