apache/beam · error

unable to dial sdk worker pool

Error message

unable to dial sdk worker pool %v: %v

What it means

externalEnvironment dials the external SDK worker pool's gRPC endpoint from the ExternalPayload. If grpc.Dial fails to establish a connection to the given URL, prism panics with this message including the endpoint URL and the underlying gRPC error.

Solutions

1. Confirm the external worker pool is running and listening on the URL in the ExternalPayload (curl/nc the host:port).
2. Fix the endpoint URL in the environment configuration (scheme, host, port).
3. Check network/firewall rules and ensure prism can reach the worker pool host.
4. Ensure credentials match (insecure vs TLS) on both runner and pool sides.

Example fix

// before
"environment_config": "{\"endpoint\":\"grpc://localhost:50001\"}" // pool actually on 50002
// after
"environment_config": "{\"endpoint\":\"grpc://localhost:50002\"}"

// Before submitting, check the external worker pool endpoint is reachable
conn, err := net.DialTimeout("tcp", hostPortFromURL(endpointURL), 5*time.Second)
if err != nil {
    return fmt.Errorf("worker pool endpoint %s unreachable: %w", endpointURL, err)
}
conn.Close()

When it happens

Trigger: Running a pipeline whose environment is an external (pre-spawned) worker pool whose endpoint URL is wrong, unreachable, or the worker pool process is not listening at that address.

Common situations: Typo in --environment_config endpoint, external worker pool crashed or never started, firewall/network policy blocking the port, or TLS/insecure credential mismatch.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• unable to start sdk worker
• chunk send failed
• empty port
• EnableHook: can't enable hook
• EnableHook: not registered

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4893e45bae8af964. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/runners/prism/internal/environments.go:129

		rankB := ranks[b.GetUrn()]

		// Reverse the comparison so our favourite is at the front
		switch {
		case rankA > rankB:
			return -1 // Usually "greater than" would be 1
		case rankA < rankB:
			return 1
		}
		return 0
	})
	// Pick our favourite.
	return envs[0]
}

func externalEnvironment(ctx context.Context, ep *pipepb.ExternalPayload, wk *worker.W) {
	conn, err := grpc.Dial(ep.GetEndpoint().GetUrl(), grpc.WithTransportCredentials(insecure.NewCredentials()))
	if err != nil {
		panic(fmt.Sprintf("unable to dial sdk worker pool %v: %v", ep.GetEndpoint().GetUrl(), err))
	}
	defer conn.Close()
	pool := fnpb.NewBeamFnExternalWorkerPoolClient(conn)

	endpoint := &pipepb.ApiServiceDescriptor{
		Url: wk.Endpoint(),
	}

	// Use a background context for these workers to avoid pre-mature
	// cancelation issues when starting them.
	bgContext := context.Background()

	resp, err := pool.StartWorker(bgContext, &fnpb.StartWorkerRequest{
		WorkerId:          wk.ID,
		ControlEndpoint:   endpoint,
		LoggingEndpoint:   endpoint,
		ArtifactEndpoint:  endpoint,
		ProvisionEndpoint: endpoint,

View on GitHub (pinned to 12126d8942)