temporalio/temporal · critical

Cannot set current time for current cluster

Error message

Cannot set current time for current cluster

What it means

This panic is fired by the history shard context's SetCurrentTime when a caller attempts to update the clock for the cluster that this shard is currently serving. The current cluster's time is owned by the shard's internal timeSource and must never be overwritten by replication, so any attempt to do so is treated as a programming-invariant violation. SetCurrentTime is only valid for remote clusters whose clock is tracked from incoming replication tasks.

Source

Thrown at service/history/shard/context_impl.go:1456

		return time.Time{}, false
	}
	// An unset visibility time would date the age from the epoch.
	if oldest := resp.Tasks[0].GetVisibilityTime(); !oldest.IsZero() {
		return oldest, true
	}
	return time.Time{}, false
}

func (s *ContextImpl) SetCurrentTime(cluster string, currentTime time.Time) {
	s.wLock()
	defer s.wUnlock()
	if cluster != s.GetClusterMetadata().GetCurrentClusterName() {
		prevTime := s.getOrUpdateRemoteClusterInfoLocked(cluster).CurrentTime
		if prevTime.Before(currentTime) {
			s.getOrUpdateRemoteClusterInfoLocked(cluster).CurrentTime = currentTime
		}
	} else {
		panic("Cannot set current time for current cluster")
	}
}

func (s *ContextImpl) GetCurrentTime(cluster string) time.Time {
	if cluster != s.GetClusterMetadata().GetCurrentClusterName() {
		s.wLock()
		defer s.wUnlock()
		return s.getOrUpdateRemoteClusterInfoLocked(cluster).CurrentTime
	}
	return s.timeSource.Now().UTC()
}

func (s *ContextImpl) getLastUpdatedTime() time.Time {
	s.rLock()
	defer s.rUnlock()
	return s.lastUpdated
}

View on GitHub (pinned to bde624efd1)

Solutions

  1. Check the caller of SetCurrentTime and skip the call when the task's cluster name equals the current cluster name (compare against GetClusterMetadata().GetCurrentClusterName()).
  2. Verify the 'current cluster name' setting in the history service config matches across all services in the ring (frontend, history, worker).
  3. If consuming replication tasks, ensure tasks for the local cluster are dropped or handled locally, never routed into the remote-clock update path.
  4. Upgrade/align all nodes to the same cluster-name configuration; a split-brain where one node thinks it is cluster A and another cluster B causes this panic.

Example fix

// before
s.shardContext.SetCurrentTime(clusterName, task.GetVisibilityTime())

// after
if clusterName != s.clusterMetadata.GetCurrentClusterName() {
    s.shardContext.SetCurrentTime(clusterName, task.GetVisibilityTime())
}
Defensive patterns

Strategy: validation

Validate before calling

if clusterName == clusterMetadata.GetCurrentClusterName() {
    return // skip SetCurrentTime for the local cluster
}
shardContext.SetCurrentTime(clusterName, t)

Prevention

When it happens

Trigger: Calling ContextImpl.SetCurrentTime(cluster, t) where cluster equals ClusterMetadata.GetCurrentClusterName(); i.e. a replication-history-task processor passes its own cluster name instead of a remote cluster name (typically a misconfigured replication queue or a cluster name sourced from the wrong metadata field).

Common situations: Multi-cluster (cross-DC) replication setups where the current cluster name is misconfigured (e.g. 'historyCfg.CurrentClusterName' in config differs between frontend/history services), or custom code that replays tasks locally without filtering out tasks originating from the local cluster.

Related errors

AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/100c9129e4810f59. Report an issue: GitHub.