{"record":{"id":"8f528d4d6257c9d1","repo":"apache/beam","slug":"failed-to-connect","errorCode":null,"errorMessage":"failed to connect","messagePattern":"failed to connect","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"sdks/go/pkg/beam/core/runtime/harness/harness.go","lineNumber":110,"sourceCode":"\t\t}\n\t\tif err := profiler.Start(cfg); err != nil {\n\t\t\tlog.Errorf(ctx, \"failed to start cloud profiler, got %v\", err)\n\t\t}\n\t}\n\n\tif tempLocation := beam.PipelineOptions.Get(\"temp_location\"); tempLocation != \"\" && samplingFrequencySeconds > 0 {\n\t\tgo diagnostics.SampleForHeapProfile(ctx, samplingFrequencySeconds, maxTimeBetweenDumpsSeconds)\n\t}\n\n\telmTimeout, err := parseTimeoutDurationFlag(ctx, beam.PipelineOptions.Get(\"element_processing_timeout\"))\n\tif err != nil {\n\t\tlog.Debugf(ctx, \"Failed to parse element_processing_timeout: %v, there will be no timeout for processing an element in a PTransform operation\", err)\n\t}\n\n\t// Connect to FnAPI control server. Receive and execute work.\n\tconn, err := dial(ctx, controlEndpoint, \"control\", 60*time.Second)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"failed to connect\")\n\t}\n\tdefer conn.Close()\n\n\tclient := fnpb.NewBeamFnControlClient(conn)\n\n\tlookupDesc := func(id bundleDescriptorID) (*fnpb.ProcessBundleDescriptor, error) {\n\t\treturn client.GetProcessBundleDescriptor(ctx, &fnpb.GetProcessBundleDescriptorRequest{ProcessBundleDescriptorId: string(id)})\n\t}\n\n\tstub, err := client.Control(ctx)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"failed to connect to control service\")\n\t}\n\n\tlog.Debugf(ctx, \"Successfully connected to control @ %v\", controlEndpoint)\n\n\t// Each ProcessBundle is a sub-graph of the original one.\n","sourceCodeStart":92,"sourceCodeEnd":128,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/core/runtime/harness/harness.go#L92-L128","documentation":"In Apache Beam Go SDK's FnAPI harness (MainWithOptions), the worker dials the control gRPC server at controlEndpoint with a 60s timeout. If dialing fails for any reason (unreachable host, TLS rejection, timeout), the underlying error is wrapped as \"failed to connect\" and worker startup aborts. This is the first network step of the Go SDK harness, so it usually reflects container networking or endpoint configuration problems.","triggerScenarios":"dial(ctx, controlEndpoint, \"control\", 60*time.Second) fails: control service address unreachable, DNS/hostname unresolvable, TLS/credentials rejected, connection refused, or the 60-second deadline expires before the gRPC channel is established.","commonSituations":"Runner-provided --control_endpoint or worker options point to a wrong host/port; the FnAPI control service container isn't up yet (startup race in Docker/Flink/Spark runners); firewall or Kubernetes NetworkPolicy blocks the port; grpc:// vs TLS implications with custom containers; endpoints behind VPC without private access.","solutions":["Verify the control endpoint host:port is correct and reachable from the worker (nc/curl from inside the worker container).","Ensure the FnAPI control/runner service is started before the SDK harness launches; tolerate startup races with retries.","Check network policy/firewall/VPC rules allow worker-to-control traffic on the gRPC port.","Re-run with debug logging to see the wrapped underlying dial error and fix its root cause (DNS, TLS, auth).","Pin/upgrade the Beam SDK and runner versions so the FnAPI endpoints and proto versions match."],"exampleFix":"// before: relying on a stale/hardcoded endpoint\nconn, err := dial(ctx, controlEndpoint, \"control\", 60*time.Second)\nif err != nil {\n    return errors.Wrap(err, \"failed to connect\")\n}\n// after: validate the endpoint before dialing\nif controlEndpoint == \"\" {\n    return errors.New(\"control endpoint not provided\")\n}\nif _, _, err := net.SplitHostPort(string(controlEndpoint)); err != nil {\n    return errors.Wrapf(err, \"invalid control endpoint %q\", controlEndpoint)\n}\nconn, err := dial(ctx, controlEndpoint, \"control\", 60*time.Second)\nif err != nil {\n    return errors.Wrap(err, \"failed to connect\")\n}","handlingStrategy":"retry","validationCode":"func validateControlEndpoint(ep string) error {\n    host, port, err := net.SplitHostPort(ep)\n    if err != nil || host == \"\" || port == \"\" {\n        return fmt.Errorf(\"invalid control endpoint %q\", ep)\n    }\n    conn, err := net.DialTimeout(\"tcp\", ep, 5*time.Second)\n    if err != nil {\n        return fmt.Errorf(\"control endpoint %q unreachable: %v\", ep, err)\n    }\n    conn.Close()\n    return nil\n}","typeGuard":null,"tryCatchPattern":"err := runHarness(ctx, opts)\nvar ne net.Error\nswitch {\ncase errors.As(err, &ne) && ne.Timeout():\n    // endpoint unreachable within 60s: check networking/service startup, retry with backoff\ncase status.Code(errors.Unwrap(err)) == codes.Unavailable:\n    // transient; retry worker startup\ndefault:\n    return err\n}","preventionTips":["Verify worker-to-control connectivity (host, port, firewall, NetworkPolicy) before launching jobs with custom containers.","Keep runner and Beam SDK versions matched so endpoints and FnAPI protos are compatible.","Make sure the control service starts before SDK workers, or tolerate initial retries.","Test custom containers locally against a runner (e.g. DirectRunner/Flink) before production jobs."],"tags":["network","grpc","beam-go","connection","startup"],"backgroundTag":"connection-refused","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}