apache/beam · error

job failed to prepare

Error message

job failed to prepare

What it means

Before submitting a job to a universal (remote) runner, Prepare calls the JobService's Prepare RPC with the pipeline, options, and job name. Any failure of this RPC is wrapped as "job failed to prepare". Preparation is the first stage of job submission, so this usually means the runner cannot accept the job at all.

Solutions

1. Check the wrapped gRPC error: if it's a connection error, verify the --endpoint host:port and that the runner service is up.
2. Verify auth/TLS settings between client and runner.
3. Confirm pipeline options are valid and supported by the runner version.
4. Retry once transient network issues are ruled out.

Example fix

// before
beam.Run(ctx, universal.New(ctx, "bad-host:8099"), p)
// after
beam.Run(ctx, universal.New(ctx, "runner.example.com:8099"), p)

// Go: verify the runner endpoint is reachable before submission
conn, err := grpc.Dial(endpoint, grpc.WithTransportCredentials(creds))
if err != nil { return fmt.Errorf("runner endpoint %s unreachable: %w", endpoint, err) }

if err := beam.Run(ctx, runner, p); err != nil && strings.Contains(err.Error(), "job failed to prepare") {
    // check endpoint, TLS, auth, and runner health before retrying
}

When it happens

Trigger: client.Prepare(ctx, req) returns an error — connection failure, auth rejection, malformed pipeline options, or server-side validation failure — when called from Execute.

Common situations: Wrong or unreachable runner endpoint (--endpoint); runner service down; TLS/auth misconfiguration; unsupported pipeline options; stale runner version that rejects the submitted pipeline proto.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• failed to submit job
• chunk send failed
• error creating local job server
• failed to close stream for
• failed to connect to state service

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/eb568af125630dd2. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/runners/universal/runnerlib/job.go:79

		Options:      beam.PipelineOptions.Export(),
		AppName:      opt.Name,
		Experiments:  append(opt.Experiments, "beam_fn_api"),
		RetainDocker: opt.RetainDocker,
		Parallelism:  opt.Parallelism,
	}

	options, err := tools.OptionsToProto(raw)
	if err != nil {
		return "", "", "", errors.WithContext(err, "producing pipeline options")
	}
	req := &jobpb.PrepareJobRequest{
		Pipeline:        p,
		PipelineOptions: options,
		JobName:         opt.Name,
	}
	resp, err := client.Prepare(ctx, req)
	if err != nil {
		return "", "", "", errors.Wrap(err, "job failed to prepare")
	}
	return resp.GetPreparationId(), resp.GetArtifactStagingEndpoint().GetUrl(), resp.GetStagingSessionToken(), nil
}

// Submit submits a job to the given job service. It returns a jobID, if successful.
func Submit(ctx context.Context, client jobpb.JobServiceClient, id, token string) (string, error) {
	req := &jobpb.RunJobRequest{
		PreparationId:  id,
		RetrievalToken: token,
	}

	resp, err := client.Run(ctx, req)
	if err != nil {
		return "", errors.Wrap(err, "failed to submit job")
	}
	return resp.GetJobId(), nil
}

View on GitHub (pinned to 12126d8942)