temporalio/temporal · error

task queue user data replication failed: %v

Error message

task queue user data replication failed: %v

What it means

ForceReplicationWorkflow signals failure when its task-queue user-data replication activity reports a FailureMessage. The workflow waits (workflow.Await) for the replication status to be Done; if the status carries a non-empty FailureMessage it surfaces it as this workflow error, marking the whole force-replication workflow as failed.

Source

Thrown at service/worker/migration/force_replication_workflow.go:192

		executionsCh.Close()
	})

	if err := enqueueReplicationTasks(ctx, executionsCh, metadataResp.NamespaceID, &params); err != nil {
		return err
	}

	if listExecutions != nil {
		return listExecutions
	}

	if params.NextPageToken == nil {
		if workflow.GetVersion(ctx, taskQueueUserDataReplicationVersionMarker, workflow.DefaultVersion, 1) > workflow.DefaultVersion {
			err := workflow.Await(ctx, func() bool { return params.TaskQueueUserDataReplicationStatus.Done })
			if err != nil {
				return err
			}
			if params.TaskQueueUserDataReplicationStatus.FailureMessage != "" {
				return fmt.Errorf("task queue user data replication failed: %v", params.TaskQueueUserDataReplicationStatus.FailureMessage)
			}
		}
		return nil
	}

	params.ContinuedAsNewCount++

	// There are still more workflows to replicate. Continue-as-new to process on a new run.
	// This prevents history size from exceeding the server-defined limit
	return workflow.NewContinueAsNewError(ctx, ForceReplicationWorkflow, params)
}

func ForceReplicationWorkflowV2(ctx workflow.Context, params ForceReplicationParams) (retErr error) {
	// For now, we'll return the initial page token for simplicity.
	// If we want this to be more precise, we could track processed pages.
	startPageToken := params.NextPageToken

	_ = workflow.SetQueryHandler(ctx, forceReplicationStatusQueryType, func() (ForceReplicationStatus, error) {

View on GitHub (pinned to bde624efd1)

Solutions

  1. Read the embedded FailureMessage in the workflow failure/result to find the underlying replication error.
  2. Re-run the force-replication workflow after fixing the root cause reported in FailureMessage.
  3. Verify the task queues and their user data exist and are valid on the target cluster.
  4. Check connectivity/auth between clusters for the task queue user data replication path.
  5. Ensure the migration worker is running a version that includes the taskQueueUserDataReplicationVersionMarker.
Defensive patterns

Strategy: try-catch

Try / catch

err := workflow.ExecuteChildWorkflow(...).Get(ctx, &out)
if err != nil {
    var appErr *temporal.ApplicationError
    if errors.As(err, &appErr) && strings.Contains(appErr.Error(), "task queue user data replication failed") {
        // inspect FailureMessage and decide whether to retry the whole workflow
    }
    return err
}

Prevention

When it happens

Trigger: During force replication with the taskQueueUserDataReplicationVersionMarker version gate passed, the child activity replicating task queue user data sets TaskQueueUserDataReplicationStatus.FailureMessage; after Await returns, the workflow returns this error.

Common situations: Task queue user data (e.g. build IDs / versioning info) failed to replicate to the target cluster because the remote call errored or the target namespace/queue is invalid; the child workflow/activity that performs the replication recorded its error message into the shared status struct.

Related errors


AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/9495ab7b2db7a990. Report an issue: GitHub.