temporalio/temporal · error

WithTokenProvider is set but no remote-cluster TLS is config

Error message

WithTokenProvider is set but no remote-cluster TLS is configured: supply global.tls.remoteClusters in config, or pass a provider via WithTLSConfigProvider

What it means

TokenCredentials for remote cluster auth must be sent over TLS (RFC 9700); the server performs a coarse startup check that a token provider is backed by some remote-cluster TLS source. If WithTokenProvider is set but neither a TLS config provider (WithTLSConfigProvider) nor global.tls.remoteClusters entries exist, this error aborts startup to avoid an unclear fatal on first cross-cluster dial.

Source

Thrown at temporal/fx.go:305

	// check that when static hosts are defined, they are defined for all required hosts
	if len(so.hostsByService) > 0 {
		for _, service := range DefaultServices {
			hosts := so.hostsByService[primitives.ServiceName(service)]
			if len(hosts.All) == 0 {
				return serverOptionsProvider{}, fmt.Errorf("%w: %v", missingServiceInStaticHosts, service)
			}
		}
	}

	if so.config.Global.Authorization.RemoteClusterAuth.Require && so.tokenProvider == nil {
		return serverOptionsProvider{}, errors.New("global.authorization.remoteClusterAuth.require is true but no TokenProvider is configured: use WithTokenProvider")
	}
	// TokenCredentials require TLS (RFC 9700); without a remote-cluster TLS source the first
	// cross-cluster dial would fatal-log, with no clear "you forgot TLS" diagnostic.
	// Coarse check: any remote-cluster TLS entry passes; per-hostname config is still validated
	// lazily on first dial.
	if so.tokenProvider != nil && so.tlsConfigProvider == nil && len(so.config.Global.TLS.RemoteClusters) == 0 {
		return serverOptionsProvider{}, errors.New("WithTokenProvider is set but no remote-cluster TLS is configured: supply global.tls.remoteClusters in config, or pass a provider via WithTLSConfigProvider")
	}

	return serverOptionsProvider{
		ServerOptions:              so,
		StopChan:                   stopChan,
		StartupSynchronizationMode: so.startupSynchronizationMode,

		Config:      so.config,
		PProfConfig: &so.config.Global.PProf,
		LogConfig:   so.config.Log,

		ServiceNames:    so.serviceNames,
		ServiceHosts:    so.hostsByService,
		NamespaceLogger: so.namespaceLogger,

		ServiceResolver:                 so.persistenceServiceResolver,
		CustomDataStoreFactory:          so.customDataStoreFactory,
		CustomVisibilityStore:           so.customVisibilityStoreFactory,

View on GitHub (pinned to bde624efd1)

Solutions

  1. Add remote cluster TLS entries under global.tls.remoteClusters in config (serverName, rootCA, etc.)
  2. Or pass a TLS source via WithTLSConfigProvider when building server options
  3. If truly intended, verify per-hostname TLS config is validated lazily on first dial and test a cross-cluster connection after boot
  4. Keep token provider and TLS configuration changes together in the same deploy to avoid mismatched states

Example fix

// before
opts := temporal.NewServerOptions(...,
    temporal.WithTokenProvider(tp))
// after
opts := temporal.NewServerOptions(...,
    temporal.WithTokenProvider(tp),
    temporal.WithTLSConfigProvider(tlsProvider))
// or in config:
// global.tls.remoteClusters:
//   clusterB: { serverName: clusterB.example.com, rootCAFile: /etc/certs/ca.pem }
Defensive patterns

Strategy: validation

Validate before calling

func validateTokenTLS(cfg *config.Config, hasTLSProvider bool) error {
    if cfg.Global.Authorization == nil || !cfg.Global.Authorization.HasTokenProviderEquivalent() { return nil }
    if hasTLSProvider || len(cfg.Global.TLS.RemoteClusters) > 0 { return nil }
    return errors.New("token provider requires remote-cluster TLS or WithTLSConfigProvider")
}

Try / catch

provider, err := temporal.ServerOptionsProvider(...)
if err != nil {
    if strings.Contains(err.Error(), "no remote-cluster TLS is configured") {
        return fmt.Errorf("supply global.tls.remoteClusters or WithTLSConfigProvider: %w", err)
    }
    return err
}

Prevention

When it happens

Trigger: Constructing server options with WithTokenProvider while Global.TLS.RemoteClusters is empty and WithTLSConfigProvider was not called.

Common situations: Enabling token auth for cross-cluster without configuring TLS per remote cluster; supplying TLS via a custom provider but forgetting WithTLSConfigProvider; partial migration from plaintext replication to secured cross-cluster links.

Understand the failure class

Related errors


AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/df3e140c58244f62. Report an issue: GitHub.