apache/beam · error

failed to stage Go worker binary

Error message

failed to stage Go worker binary: %v

What it means

stageFiles wraps the error from stageFile when staging the prebuilt Go worker binary (URN graphx.URNArtifactGoWorker) fails on the staging stream. An io.EOF is intentionally not wrapped — it falls through to get the real stream error — so this error means a real send/open failure while uploading the binary.

Solutions

1. Ensure the Go worker binary is built (go build) and the correct path is passed
2. Check os.Open failure in the wrapped error (file-not-found vs permission)
3. Retry on transient gRPC errors; StageViaPortableAPI already retries at a higher level
4. Increase timeouts for large binaries over slow links

Example fix

// before
return errors.Wrapf(err, "failed to stage Go worker binary: %v", binary)
// after: build first, e.g.
// go build -o worker/main worker/main.go
// ./submit --workerBinary=worker/main ...

if _, err := os.Stat(workerBinary); err != nil {
  return fmt.Errorf("worker binary missing at %s: %w", workerBinary, err)
}

if err := submit(...); err != nil && strings.Contains(err.Error(), "failed to stage Go worker binary") {
  // check os.Open failure in wrapped err: path wrong or binary not built
}

When it happens

Trigger: stageFile(binary, stream) returns a non-EOF error: os.Open failed on the worker binary, or a stream.Send of file chunks failed mid-upload.

Common situations: Worker binary not built before submission (go build skipped), binary path wrong due to --workerBinary flag mistake, connection dropped during a large binary upload.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• failed to stage file
• chunk send failed
• expected header as first message
• failed to close stream for
• failed to receive header

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/76cdb77a42d6164c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/runners/universal/runnerlib/stage.go:119

				Response: &jobpb.ArtifactResponseWrapper_ResolveArtifactResponse{
					ResolveArtifactResponse: &jobpb.ResolveArtifactsResponse{
						Replacements: request.ResolveArtifact.Artifacts,
					},
				}})
			if err != nil {
				return err
			}

		case *jobpb.ArtifactRequestWrapper_GetArtifact:
			switch typeUrn := request.GetArtifact.Artifact.TypeUrn; typeUrn {
			// TODO(https://github.com/apache/beam/issues/21459): Legacy Type URN. If requested, provide the binary.
			// To be removed later in 2022, once thoroughly obsolete.
			case graphx.URNArtifactGoWorker:
				if err := stageFile(binary, stream); err != nil {
					if err == io.EOF {
						continue // so we can get the real error from stream.Recv.
					}
					return errors.Wrapf(err, "failed to stage Go worker binary: %v", binary)
				}
			case graphx.URNArtifactFileType:
				typePl := pipepb.ArtifactFilePayload{}
				if err := proto.Unmarshal(request.GetArtifact.Artifact.TypePayload, &typePl); err != nil {
					return errors.Wrap(err, "failed to parse artifact file payload")
				}
				if err := stageFile(typePl.GetPath(), stream); err != nil {
					if err == io.EOF {
						continue // so we can get the real error from stream.Recv.
					}
					return errors.Wrapf(err, "failed to stage file %v", typePl.GetPath())

				}
			default:
				return errors.Errorf("request has unexpected artifact type %s", typeUrn)
			}

		default:

View on GitHub (pinned to 12126d8942)