temporalio/temporal · error

unable to list Nexus endpoints for namespace %s: %w

Error message

unable to list Nexus endpoints for namespace %s: %w

What it means

ValidateNexusEndpointsActivity lists Nexus endpoints page by page from persistence to verify the namespace being deleted is not a target of any endpoint. If the persistence ListNexusEndpoints call fails, the activity logs the failure and wraps it in this error, aborting namespace deletion validation.

Source

Thrown at service/worker/deletenamespace/activities.go:127

	}
	return nil
}

func (a *localActivities) ValidateNexusEndpointsActivity(ctx context.Context, nsID namespace.ID, nsName namespace.Name) error {
	if a.allowDeleteNamespaceIfNexusEndpointTarget() {
		return nil
	}
	// Prevent deletion of a namespace that is targeted by a Nexus endpoint.
	var nextPageToken []byte
	for {
		resp, err := a.nexusEndpointManager.ListNexusEndpoints(ctx, &persistence.ListNexusEndpointsRequest{
			LastKnownTableVersion: 0,
			NextPageToken:         nextPageToken,
			PageSize:              a.nexusEndpointListDefaultPageSize(),
		})
		if err != nil {
			a.logger.Error("Unable to list Nexus endpoints from persistence.", tag.WorkflowNamespace(nsName.String()), tag.WorkflowNamespaceID(nsID.String()), tag.Error(err))
			return fmt.Errorf("unable to list Nexus endpoints for namespace %s: %w", nsName, err)
		}

		for _, entry := range resp.Entries {
			if endpointNsID := entry.GetEndpoint().GetSpec().GetTarget().GetWorker().GetNamespaceId(); endpointNsID == nsID.String() {
				return errors.NewFailedPrecondition(fmt.Sprintf("cannot delete a namespace that is a target of a Nexus endpoint %s", entry.GetEndpoint().GetSpec().GetName()), nil)
			}
		}
		nextPageToken = resp.NextPageToken
		if len(nextPageToken) == 0 {
			break
		}
	}
	return nil
}

func (a *localActivities) MarkNamespaceDeletedActivity(ctx context.Context, nsName namespace.Name) error {
	ctx = headers.SetCallerName(ctx, nsName.String())

View on GitHub (pinned to bde624efd1)

Solutions

  1. Retry the namespace deletion activity once persistence is healthy — the error is typically transient.
  2. Check the wrapped persistence error (and logged tag.Error) to diagnose DB connectivity, timeouts, or schema issues.
  3. Verify persistence configuration and endpoint table health (LastKnownTableVersion handling, migrations applied).
  4. Increase activity retry policy attempts/interval for this activity so transient DB blips don't fail the workflow.

Example fix

// before
activityOpts := workflow.ActivityOptions{StartToCloseTimeout: 10 * time.Second} // too tight, times out listing endpoints
// after
activityOpts := workflow.ActivityOptions{
  StartToCloseTimeout: 30 * time.Second,
  RetryPolicy: &temporal.RetryPolicy{InitialInterval: time.Second, MaximumAttempts: 5},
}
Defensive patterns

Strategy: retry

Validate before calling

// pre-check persistence health before running the deletion workflow
if err := persistenceHealthCheck(ctx); err != nil {
  return fmt.Errorf("postpone namespace deletion; persistence unhealthy: %w", err)
}

Try / catch

err := workflow.ExecuteActivity(ctx, a.ValidateNexusEndpointsActivity, nsName, nsID).Get(ctx, nil)
if err != nil {
  if strings.Contains(err.Error(), "unable to list Nexus endpoints") {
    logger.Warn("transient persistence failure validating nexus endpoints; will retry", tag.Error(err))
    return err // let workflow retry policy handle it
  }
  return err
}

Prevention

When it happens

Trigger: Deleting a namespace when the persistence-backed ListNexusEndpoints call errors — DB unavailable, timeout, shard/row issues, or persistence connectivity problems during the paginated loop.

Common situations: Database outage or high latency during namespace deletion; persistence retries exhausted; cluster under load causing list timeouts; running the delete-Nexus-validation activity against a misconfigured persistence store.

Related errors


AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/d7e9fa1dd5d59602. Report an issue: GitHub.