temporalio/temporal · warning

cannot resolve Nexus endpoints partition owner: %w

Error message

cannot resolve Nexus endpoints partition owner: %w

What it means

The matching engine periodically checks whether it still owns the Nexus endpoints table partition by resolving the partition's current owner through the service resolver. This error wraps the failure of that ring/membership Lookup call for the nexusEndpointsTablePartitionRoutingKey, meaning the engine could not determine which host owns the partition. The caller treats this as 'not the owner' and aborts the ownership check.

Source

Thrown at service/matching/matching_engine.go:2927

			case <-ctx.Done():
				return resp, nil
			case <-tableVersionChanged:
				continue
			}
		}

		return resp, err
	}
}

func (e *matchingEngineImpl) checkNexusEndpointsOwnership() (bool, <-chan struct{}, error) {
	// Get the channel before checking the condition to prevent the channel from being closed while we're running this
	// check.
	ch := e.nexusEndpointsOwnershipLostCh.Load().(chan struct{}) //nolint:revive // type is always chan struct{}
	self := e.hostInfoProvider.HostInfo().Identity()
	owner, err := e.serviceResolver.Lookup(nexusEndpointsTablePartitionRoutingKey)
	if err != nil {
		return false, nil, fmt.Errorf("cannot resolve Nexus endpoints partition owner: %w", err)
	}
	return owner.Identity() == self, ch, nil
}

func (e *matchingEngineImpl) notifyNexusEndpointsOwnershipChange() {
	// We don't care about the channel returned here. This method is ensured to only be called from the single
	// watchMembership method and is the only way the channel may be replaced.
	isOwner, _, err := e.checkNexusEndpointsOwnership()
	if err != nil {
		e.logger.Error("Failed to check Nexus endpoints ownership", tag.Error(err))
		return
	}
	if !isOwner {
		close(e.nexusEndpointsOwnershipLostCh.Swap(make(chan struct{})).(chan struct{})) //nolint:revive // type is always chan struct{}
	}
	e.nexusEndpointClient.notifyOwnershipChanged(isOwner)
}

View on GitHub (pinned to bde624efd1)

Solutions

1. Retry the operation: the ownership check runs periodically, and a transient membership lookup failure typically self-heals on the next tick
2. Verify membership service health and that the service resolver's ring is populated (check membership provider logs/membership endpoint)
3. Check internal-frontend / service peer configuration and network connectivity between matching and the resolver target
4. If persistent after stable membership, restart the affected matching host so it re-registers with the ring

// Before relying on nexus endpoint ownership, confirm the resolver is reachable
owner, err := serviceResolver.Lookup(nexusEndpointsTablePartitionRoutingKey)
if err != nil { return err } // surface membership problems early

for attempt := 0; attempt < 3; attempt++ {
	isOwner, _, err := checkOwnership(ctx)
	if err == nil {
		break
	}
	logger.Warn("nexus endpoints ownership lookup failed, retrying", tag.Error(err))
	time.Sleep(backoff(attempt))
}

When it happens

Trigger: e.isNexusEndpointsTableOwner (via the shown helper) runs while the membership service resolver fails to resolve the routing key — e.g. membership provider errors, ring not yet populated, or internal frontend/service discovery unavailable.

Common situations: Startup before membership/ring convergence; a membership provider hiccup or network partition; host membership churn (many joining/leaving hosts) making the resolver temporarily unable to return an owner; misconfigured internal frontend address.

Related errors

• error loading nexus endpoints cache: %w
• error loading nexus endpoint cache: %w
• nexus operation processor failed
• cannot serialize HSM task. unable to cast to expected type
• completion token has no operation state machine reference

AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/ec8661feefeb09a9. Report an issue: GitHub.