{"record":{"id":"ad026f348ff44e5e","repo":"temporalio/temporal","slug":"failed-to-start-service-v-w","errorCode":null,"errorMessage":"failed to start service %v: %w","messagePattern":"failed to start service (.+?): %w","errorType":"console","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"temporal/server_impl.go","lineNumber":142,"sourceCode":"}\n\nfunc (s *ServerImpl) startServices() error {\n\t// The membership join time may exceed the configured max join duration.\n\t// Double the service start timeout to make sure there is enough time for start logic.\n\ttimeout := max(serviceStartTimeout, 2*s.so.config.Global.Membership.MaxJoinDuration)\n\tctx, cancel := context.WithTimeout(context.Background(), timeout)\n\tdefer cancel()\n\n\tsvcs := slices.Clone(s.servicesMetadata)\n\tslices.SortFunc(svcs, func(a, b *ServicesMetadata) int {\n\t\treturn cmp.Compare(initOrder[a.serviceName], initOrder[b.serviceName])\n\t})\n\n\tvar allErrs error\n\tfor _, svc := range svcs {\n\t\terr := svc.app.Start(ctx)\n\t\tif err != nil {\n\t\t\tallErrs = multierr.Append(allErrs, fmt.Errorf(\"failed to start service %v: %w\", svc.serviceName, err))\n\t\t}\n\t}\n\treturn allErrs\n}\n\nfunc initSystemNamespaces(\n\tctx context.Context,\n\tcfg *config.Persistence,\n\tcurrentClusterName string,\n\tpersistenceServiceResolver resolver.ServiceResolver,\n\tpersistenceFactoryProvider persistenceClient.FactoryProviderFn,\n\tlogger log.Logger,\n\tcustomDataStoreFactory persistenceClient.AbstractDataStoreFactory,\n\tmetricsHandler metrics.Handler,\n\tserializer serialization.Serializer,\n) error {\n\tclusterName := persistenceClient.ClusterName(currentClusterName)\n\tmetricsHandler = metricsHandler.WithTags(metrics.ServiceNameTag(primitives.ServerService))","sourceCodeStart":124,"sourceCodeEnd":160,"githubUrl":"https://github.com/temporalio/temporal/blob/bde624efd13fbd3843654058db6d9c716166318b/temporal/server_impl.go#L124-L160","documentation":"Wraps errors returned by the Start method of each internal Temporal service application (frontend, history, matching, worker, internal-frontend). startServices iterates services in init order and collects every failure into a multierr, so one bad service does not stop the others from attempting to start. The %w cause carries the service-specific error (ringpop membership join failure, gRPC listener bind error, etc.).","triggerScenarios":"temporal.Server.Start(ctx) -> startServices, when any service's app.Start(ctx) returns an error: failing to bind its gRPC port, failing to join the membership (ringpop/membership host:port unreachable), failing to initialize the service's own persistence/visibility clients, or exceeding the start timeout (max(serviceStartTimeout, 2*MaxJoinDuration)).","commonSituations":"Port already in use (another Temporal process or stale container holds the frontend/history/matching port); membership advertised IP unreachable in containerized/K8s setups (wrong broadcast host); multiple services sharing a config that points at an unavailable dependency; partial cluster outage where one service can't reach the DB or Elasticsearch.","solutions":["Read the wrapped cause and the service name in the message to identify which service and what sub-error failed","Check that all service ports (e.g. 7233/7234/7235/7239 per config) are free: lsof/ss on the host or a stale container","Verify membership config (broadcast host, host:port list) is reachable from every node — a common cause in Docker/Kubernetes","Check the shared dependencies of the failing service (DB, Elasticsearch, dynamicconfig source) are up","For multi-error output, note multierr appends all failures — fix the root shared dependency if many services failed"],"exampleFix":"// before: config serves frontend on default 7233 but an old process still holds it\n// failed to start service frontend: listen tcp :7233: bind: address already in use\n\n// after: stop the stale process, then start\n// kill <stale-pid>\nif err := srv.Start(ctx); err != nil {\n\tlog.Fatalf(\"server failed: %v\", err)\n}","handlingStrategy":"try-catch","validationCode":"// pre-flight port availability check\n// for _, p := range []int{7233, 7234, 7235, 7239} {\n// \tln, err := net.Listen(\"tcp\", fmt.Sprintf(\":%d\", p))\n// \tif err != nil { log.Fatalf(\"port %d in use\", p) }\n// \tln.Close()\n// }","typeGuard":null,"tryCatchPattern":"if err := srv.Start(ctx); err != nil {\n\tlog.Fatalf(\"one or more services failed to start: %v\", err)\n\t// message lists every failing service via multierr —\n\t// address the shared root cause if multiple services failed\n}","preventionTips":["Check service ports are free before launch (no stale processes/containers)","Validate membership broadcast host is routable in Docker/K8s","Ensure DB and visibility store are healthy before starting services","Give services enough MaxJoinDuration on slow networks","Start services one at a time when debugging to isolate the failing one"],"tags":["startup","services","grpc","membership"],"backgroundTag":"service-start-failed","analyzedSha":"bde624efd13fbd3843654058db6d9c716166318b","analyzedAt":"2026-09-01T07:18:39.080Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}