apache/beam · error

prism didn't get control connection to after

Error message

prism %v didn't get control connection to %v after %v

What it means

After environments start, RunPipeline schedules a 1-minute timeout that checks whether the worker connected to prism's control endpoint. If the worker is neither Connected nor Stopped by then, the job is failed and cancelled with this error naming the worker and its endpoint.

Solutions

1. Check harness container logs (`docker logs <container>`) for startup crashes
2. Ensure the prism job endpoint is reachable from the container (use host gateway address, not localhost)
3. Verify SDK harness image version matches the pipeline's SDK version
4. Confirm no firewall blocks the control port; restart prism if the port is stale

Example fix

// before: prism advertises localhost inside container
// after: bind endpoint to a docker-reachable address
prism --address 0.0.0.0:8073 --endpoint host.docker.internal:8073

conn, err := net.DialTimeout("tcp", endpoint, 5*time.Second)
if err != nil {
  return fmt.Errorf("control endpoint %s unreachable before submit: %w", endpoint, err)
}
conn.Close()

if strings.Contains(err.Error(), "didn't get control connection") {
  // check harness logs and endpoint reachability, then retry
}

When it happens

Trigger: The SDK harness container/process started but never established the control RPC connection within 60 seconds — wrong endpoint advertised, harness crash after start, or network/firewall blocking the port.

Common situations: Harness image version mismatched with pipeline, container booting slowly or crashing, prism endpoint unreachable from inside Docker (e.g. localhost vs host address), or heavy startup delays.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• couldn't connect to docker
• environment with urn unimplemented
• failed to start environment
• forceLpCoders: coder
• lpUnknownCoders: coder

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ca4ce47764a78367. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/runners/prism/internal/execute.go:67

	// environments, and start up docker containers, but
	// here, we only want and need the go one, operating
	// in loopback mode.
	envs := j.Pipeline.GetComponents().GetEnvironments()
	wks := map[string]*worker.W{}
	for envID := range envs {
		wk := j.MakeWorker(envID)
		wks[envID] = wk
		if err := runEnvironment(j.RootCtx, j, envID, wk); err != nil {
			j.Failed(fmt.Errorf("failed to start environment %v for job %v: %w", envID, j, err))
			return
		}
		// Check for connection succeeding after we've created the environment successfully.
		timeout := 1 * time.Minute
		time.AfterFunc(timeout, func() {
			if wk.Connected() || wk.Stopped() {
				return
			}
			err := fmt.Errorf("prism %v didn't get control connection to %v after %v", wk, wk.Endpoint(), timeout)
			j.Failed(err)
			j.CancelFn(err)
		})
	}

	// When this function exits, we cancel the context to clear
	// any related job resources.
	defer func() {
		j.CancelFn(fmt.Errorf("runPipeline returned, cleaning up"))
		j.WaitForCleanUp()
	}()

	// Add this defer function to capture and log panics.
	defer func() {
		if e := recover(); e != nil {
			j.Failed(fmt.Errorf("pipeline panicked: %v\nStacktrace: %s", e, string(debug.Stack())))
		}
	}()

View on GitHub (pinned to 12126d8942)