What causes "failed to start service %v: %w" in temporalio/temporal?

Read the wrapped cause and the service name in the message to identify which service and what sub-error failed

failed to start service %v: %w

Error message

failed to start service %v: %w

What it means

Wraps errors returned by the Start method of each internal Temporal service application (frontend, history, matching, worker, internal-frontend). startServices iterates services in init order and collects every failure into a multierr, so one bad service does not stop the others from attempting to start. The %w cause carries the service-specific error (ringpop membership join failure, gRPC listener bind error, etc.).

Source

Thrown at temporal/server_impl.go:142

}

func (s *ServerImpl) startServices() error {
	// The membership join time may exceed the configured max join duration.
	// Double the service start timeout to make sure there is enough time for start logic.
	timeout := max(serviceStartTimeout, 2*s.so.config.Global.Membership.MaxJoinDuration)
	ctx, cancel := context.WithTimeout(context.Background(), timeout)
	defer cancel()

	svcs := slices.Clone(s.servicesMetadata)
	slices.SortFunc(svcs, func(a, b *ServicesMetadata) int {
		return cmp.Compare(initOrder[a.serviceName], initOrder[b.serviceName])
	})

	var allErrs error
	for _, svc := range svcs {
		err := svc.app.Start(ctx)
		if err != nil {
			allErrs = multierr.Append(allErrs, fmt.Errorf("failed to start service %v: %w", svc.serviceName, err))
		}
	}
	return allErrs
}

func initSystemNamespaces(
	ctx context.Context,
	cfg *config.Persistence,
	currentClusterName string,
	persistenceServiceResolver resolver.ServiceResolver,
	persistenceFactoryProvider persistenceClient.FactoryProviderFn,
	logger log.Logger,
	customDataStoreFactory persistenceClient.AbstractDataStoreFactory,
	metricsHandler metrics.Handler,
	serializer serialization.Serializer,
) error {
	clusterName := persistenceClient.ClusterName(currentClusterName)
	metricsHandler = metricsHandler.WithTags(metrics.ServiceNameTag(primitives.ServerService))

View on GitHub (pinned to bde624efd1)

Solutions

Read the wrapped cause and the service name in the message to identify which service and what sub-error failed
Check that all service ports (e.g. 7233/7234/7235/7239 per config) are free: lsof/ss on the host or a stale container
Verify membership config (broadcast host, host:port list) is reachable from every node — a common cause in Docker/Kubernetes
Check the shared dependencies of the failing service (DB, Elasticsearch, dynamicconfig source) are up
For multi-error output, note multierr appends all failures — fix the root shared dependency if many services failed

Example fix

// before: config serves frontend on default 7233 but an old process still holds it
// failed to start service frontend: listen tcp :7233: bind: address already in use

// after: stop the stale process, then start
// kill <stale-pid>
if err := srv.Start(ctx); err != nil {
	log.Fatalf("server failed: %v", err)
}

Defensive patterns

Strategy: try-catch

Validate before calling

// pre-flight port availability check
// for _, p := range []int{7233, 7234, 7235, 7239} {
// 	ln, err := net.Listen("tcp", fmt.Sprintf(":%d", p))
// 	if err != nil { log.Fatalf("port %d in use", p) }
// 	ln.Close()
// }

Try / catch

if err := srv.Start(ctx); err != nil {
	log.Fatalf("one or more services failed to start: %v", err)
	// message lists every failing service via multierr —
	// address the shared root cause if multiple services failed
}

Prevention

Check service ports are free before launch (no stale processes/containers)
Validate membership broadcast host is routable in Docker/K8s
Ensure DB and visibility store are healthy before starting services
Give services enough MaxJoinDuration on slow networks
Start services one at a time when debugging to isolate the failing one

When it happens

Trigger: temporal.Server.Start(ctx) -> startServices, when any service's app.Start(ctx) returns an error: failing to bind its gRPC port, failing to join the membership (ringpop/membership host:port unreachable), failing to initialize the service's own persistence/visibility clients, or exceeding the start timeout (max(serviceStartTimeout, 2*MaxJoinDuration)).

Common situations: Port already in use (another Temporal process or stale container holds the frontend/history/matching port); membership advertised IP unreachable in containerized/K8s setups (wrong broadcast host); multiple services sharing a config that points at an unavailable dependency; partial cluster outage where one service can't reach the DB or Elasticsearch.

Related errors

AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/ad026f348ff44e5e. Report an issue: GitHub.