What causes "error loading nexus endpoints cache: %w" in temporalio/temporal?

Retry the CreateNexusEndpoint call once the database/persistence layer is reachable — the cache load is retried on the next call

temporalio/temporal · error

error loading nexus endpoints cache: %w

Error message

error loading nexus endpoints cache: %w

What it means

CreateNexusEndpoint must load the entire Nexus endpoints table into the matching engine's in-memory cache before creating a new endpoint, both to enforce name-uniqueness and to know the last table version for a conditional persistence update. This error wraps any failure of that initial loadEndpoints call (persistence read, version conflicts, context cancellation).

Source

Thrown at service/matching/nexus_endpoint_client.go:92

	endpointsRefreshInterval dynamicconfig.DurationPropertyFn,
	persistence p.NexusEndpointManager,
) *nexusEndpointClient {
	return &nexusEndpointClient{
		endpointsRefreshInterval: endpointsRefreshInterval,
		persistence:              persistence,
		tableVersionChanged:      make(chan struct{}),
	}
}

func (m *nexusEndpointClient) CreateNexusEndpoint(
	ctx context.Context,
	request *internalCreateNexusEndpointRequest,
) (*matchingservice.CreateNexusEndpointResponse, error) {
	if !m.hasLoadedEndpoints.Load() {
		// Endpoints must be loaded into memory before Create so we know whether this endpoint name is in use and that we
		// have the last known table version to update persistence.
		if err := m.loadEndpoints(ctx); err != nil {
			return nil, fmt.Errorf("error loading nexus endpoints cache: %w", err)
		}
	}

	m.Lock()
	defer m.Unlock()

	if _, exists := m.endpointsByName[request.spec.GetName()]; exists {
		return nil, serviceerror.NewAlreadyExistsf("error creating Nexus endpoint. Endpoint with name %v already registered", request.spec.GetName())
	}

	entry := &persistencespb.NexusEndpointEntry{
		Version: 0,
		Id:      uuid.NewString(),
		Endpoint: &persistencespb.NexusEndpoint{
			Clock:       hlc.Zero(request.clusterID),
			Spec:        request.spec,
			CreatedTime: timestamppb.New(request.timeSource.Now().UTC()),
		},

View on GitHub (pinned to bde624efd1)

Solutions

Retry the CreateNexusEndpoint call once the database/persistence layer is reachable — the cache load is retried on the next call
Inspect the wrapped cause (%w) for persistence errors and fix the underlying DB issue (connectivity, permissions, timeouts)
Increase the client request timeout if loadEndpoints times out on large tables
Check matching service logs for repeated loadEndpoints failures indicating schema or shard issues

Defensive patterns

Strategy: retry

Try / catch

resp, err := client.MatchingClient().CreateNexusEndpoint(ctx, req)
if err != nil {
	var persistenceErr *persistence.OperationFailedError // inspect cause via errors.Unwrap
	if errors.Is(err, context.DeadlineExceeded) || isTransientDBError(err) {
		// retry after backoff
	}
	return fmt.Errorf("create nexus endpoint failed: %w", err)
}

Prevention

Ensure the visibility/persistence DB is healthy before running nexus endpoint admin operations
Set generous client timeouts — the first call after startup pays the full table-load cost
Retry idempotently: create failures during cache load are safe to retry
Check for name collisions before creating to avoid unrelated create failures

When it happens

Trigger: Calling the matching service CreateNexusEndpoint API when hasLoadedEndpoints is false and loadEndpoints fails — typically a persistence/DB error reading the nexus endpoints table or a cancelled/deadlined context.

Common situations: Database outage or degradation on the first endpoint operation after matching service start; context deadline exceeded while loading a large endpoints table; ops tooling (tctl nexus endpoint create) hitting an unhealthy DB.

Related errors

AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/53ba1d6768c28695. Report an issue: GitHub.