apache/beam · error

unable to start sdk worker

Error message

unable to start sdk worker %v error: %v, resp: %v

What it means

After dialing the external worker pool, externalEnvironment calls PrepareJob on the BeamFnExternalWorkerPoolClient. If the RPC returns an error or the response carries an error string, prism panics with the endpoint, the RPC error, and the prototext-formatted response.

Solutions

1. Read the resp error text in the panic; it usually states why PrepareJob failed.
2. Verify environment params (e.g. SDK version, worker pool config) match what the pool expects.
3. Check the worker pool's own logs for the underlying launch failure.
4. Upgrade runner and SDK to matching versions so the external pool API is compatible.

// Validate params before PrepareJob
if len(params) == 0 || sdkVersion == "" {
    return errors.New("environment params and SDK version required by external pool")
}

resp, err := pool.PrepareJob(ctx, req)
if err != nil {
    return fmt.Errorf("PrepareJob failed: %w", err)
}
if s := resp.GetError(); s != "" {
    return fmt.Errorf("pool rejected job: %s", s)
}

When it happens

Trigger: The external worker pool rejects the PrepareJob request: bad provisioning parameters, incompatible worker pool, artifact endpoint misconfiguration, or the pool side fails to start the SDK worker process.

Common situations: Version mismatch between runner and SDK harness worker pool, missing required params in environment_config, worker pool internal failure launching SDK workers, or artifact staging endpoint unreachable from the pool.

Related errors

• unable to dial sdk worker pool
• chunk send failed
• couldn't decode characteristic for variant
• Creating CustomCoder for type failed
• empty port

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/39c00e6c30ae420a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/runners/prism/internal/environments.go:152

	endpoint := &pipepb.ApiServiceDescriptor{
		Url: wk.Endpoint(),
	}

	// Use a background context for these workers to avoid pre-mature
	// cancelation issues when starting them.
	bgContext := context.Background()

	resp, err := pool.StartWorker(bgContext, &fnpb.StartWorkerRequest{
		WorkerId:          wk.ID,
		ControlEndpoint:   endpoint,
		LoggingEndpoint:   endpoint,
		ArtifactEndpoint:  endpoint,
		ProvisionEndpoint: endpoint,
		Params:            ep.GetParams(),
	})

	if str := resp.GetError(); err != nil || str != "" {
		panic(fmt.Sprintf("unable to start sdk worker %v error: %v, resp: %v", ep.GetEndpoint().GetUrl(), err, prototext.Format(resp)))
	}

	// Job processing happens here, but orchestrated by other goroutines
	// This goroutine blocks until the context is cancelled, signalling
	// that the pool runner should stop the worker.
	<-ctx.Done()

	// Previous context cancelled so we need a new one
	// for this request.
	pool.StopWorker(bgContext, &fnpb.StopWorkerRequest{
		WorkerId: wk.ID,
	})
	wk.Stop()
}

func dockerEnvironment(ctx context.Context, logger *slog.Logger, dp *pipepb.DockerPayload, wk *worker.W, artifactEndpoint string) error {
	logger = logger.With("worker_id", wk.ID, "image", dp.GetContainerImage())

View on GitHub (pinned to 12126d8942)