What causes "failed to get job stream" in apache/beam?

Check the wrapped gRPC error for connectivity vs. job-id issues.

failed to get job stream

Error message

failed to get job stream

What it means

WaitForCompletion monitors a running job by opening the runner's GetMessageStream gRPC stream. If opening the stream fails, the error is wrapped as "failed to get job stream". Without the message stream, the client cannot observe state changes or logs for the job.

Source

Thrown at sdks/go/pkg/beam/runners/universal/runnerlib/job.go:103

func Submit(ctx context.Context, client jobpb.JobServiceClient, id, token string) (string, error) {
	req := &jobpb.RunJobRequest{
		PreparationId:  id,
		RetrievalToken: token,
	}

	resp, err := client.Run(ctx, req)
	if err != nil {
		return "", errors.Wrap(err, "failed to submit job")
	}
	return resp.GetJobId(), nil
}

// WaitForCompletion monitors the given job until completion. It logs any messages
// and state changes received.
func WaitForCompletion(ctx context.Context, client jobpb.JobServiceClient, jobID string) error {
	stream, err := client.GetMessageStream(ctx, &jobpb.JobMessagesRequest{JobId: jobID})
	if err != nil {
		return errors.Wrap(err, "failed to get job stream")
	}

	mostRecentError := "<no error received>"
	var errReceived, jobFailed bool

	for {
		msg, err := stream.Recv()
		if err != nil {
			if err == io.EOF {
				if jobFailed {
					// Connection finished, so time to exit, produce what we have.
					return errors.Errorf("job %v failed:\n%v", jobID, mostRecentError)
				}
				return nil
			}
			return err
		}

View on GitHub (pinned to 12126d8942)

Solutions

Check the wrapped gRPC error for connectivity vs. job-id issues.
Verify the runner endpoint and TLS settings.
Check the job's state on the runner (it may still be running without a message stream).
Retry the job monitoring stream or resubmit the job.

Defensive patterns

Strategy: retry

Try / catch

if err := beam.Run(ctx, runner, p); err != nil && strings.Contains(err.Error(), "failed to get job stream") {
    // check job state via GetJobState RPC or the runner UI before resubmitting
}

Prevention

Ensure a stable network connection to the runner during job execution.
Avoid aggressive context deadlines that can kill long-lived message streams.
Verify TLS/proxy settings that may break gRPC streaming.
Check job existence on the runner if the id may be stale.

When it happens

Trigger: client.GetMessageStream(ctx, &jobpb.JobMessagesRequest{JobId: jobID}) returns an error immediately after job submission, called from Execute.

Common situations: Network interruption right after submission; runner rejects the job id; gRPC/TLS misconfiguration; runner restarts between Submit and WaitForCompletion.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6f14e97143cfec8d. Report an issue: GitHub.