apache/beam · error

error connecting to job server at %v: %v

Error message

error connecting to job server at %v: %v

What it means

prism's makeJobClient connects to the Beam job service endpoint given via --endpoint. When grpc.DialContext with WithBlock fails (endpoint unreachable, DNS failure, TLS/context timeout), the command wraps the cause in "error connecting to job server at %v: %v" and exits. With no endpoint, prism instead starts its own in-process job server.

Source

Thrown at sdks/go/cmd/prism/prism.go:77

		},
		*jobManagerEndpoint)
	if err != nil {
		log.Fatalf("error creating job server: %v", err)
	}
	if *serveHTTP {
		if err := prism.CreateWebServer(ctx, cli, prism.Options{Port: *webPort}); err != nil {
			log.Fatalf("error creating web server: %v", err)
		}
	}
	// Block main thread forever to keep main from exiting.
	<-ctx.Done()
}

func makeJobClient(ctx context.Context, opts prism.Options, endpoint string) (jobpb.JobServiceClient, error) {
	if endpoint != "" {
		clientConn, err := grpc.DialContext(ctx, endpoint, grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithBlock())
		if err != nil {
			return nil, fmt.Errorf("error connecting to job server at %v: %v", endpoint, err)
		}
		return jobpb.NewJobServiceClient(clientConn), nil
	}
	cli, err := prism.CreateJobServer(ctx, opts)
	if err != nil {
		return nil, fmt.Errorf("error creating local job server: %v", err)
	}
	return cli, nil
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Verify the job server is running and the address matches: `nc -vz <host> <port>` or `grpcurl` the endpoint before submitting.
2. If no external server is needed, drop the --endpoint flag so prism creates its own local job server via prism.CreateJobServer.
3. Fix interface/binding mismatch: bind the server on 0.0.0.0 (or the reachable host) and use the published container port when submitting from the host.
4. Check the wrapped cause in the message for the root error (connection refused vs. context deadline) and increase the deadline if the server is slow to accept.

Example fix

// before
$ prism --endpoint localhost:8073 // nothing listening
// error connecting to job server at localhost:8073: ...

// after
$ prism serve --job-port 8073 &
$ prism --endpoint localhost:8073 // or omit --endpoint entirely

// before dialing
if timeout, err := net.DialTimeout("tcp", endpoint, 2*time.Second); err != nil {
    return fmt.Errorf("endpoint %s not reachable: %v", endpoint, err)
} else { timeout.Close() }

cli, err := makeJobClient(ctx, opts, endpoint)
if err != nil {
    log.Fatalf("job server setup failed: %v", err) // cause is wrapped; inspect %v chain
}

When it happens

Trigger: Running `go run ./sdks/go/cmd/prism --endpoint <addr>` where <addr> is not listening (prism/job server not started), wrong host/port, the endpoint only bound to a different interface, or the context deadline expires before the gRPC connection becomes ready.

Common situations: Pointing --endpoint at a server that was never started or already exited; typos in host:port or stale container/k8s service address; firewall/network policy blocking the port; submitting from outside Docker to a server bound inside a container (use host.docker.internal / published ports); --endpoint set while intending in-process prism (omit it to auto-create a local server).

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• Channel closed prematurely.
• failed to connect to data service at %v
• failed to connect
• unable to connect to expansion service at %v
• failed to dial server at %v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/70ece3fb07b73938. Report an issue: GitHub.