temporalio/temporal · critical

Unknown repication task type: %v

Error message

Unknown repication task type: %v

What it means

readMessagesWithAckLevel converts tasks read from a replication DLQ into ReplicationTaskInfo protobufs; the switch handles SyncActivity, HistoryReplication, SyncWorkflowState and SyncHSM task types. Any other concrete task type in the DLQ is unsupported by this handler and panics. It guards against new task types being written to the DLQ without updating DLQ read logic.

Source

Thrown at service/history/replication/dlq_handler.go:293

				WorkflowId:       task.WorkflowID,
				RunId:            task.RunID,
				TaskType:         enumsspb.TASK_TYPE_REPLICATION_SYNC_WORKFLOW_STATE,
				TaskId:           task.TaskID,
				Version:          task.Version,
				FirstEventId:     0,
				NextEventId:      0,
				ScheduledEventId: 0,
			})
		case *tasks.SyncHSMTask:
			taskInfo = append(taskInfo, &replicationspb.ReplicationTaskInfo{
				NamespaceId: task.NamespaceID,
				WorkflowId:  task.WorkflowID,
				RunId:       task.RunID,
				TaskType:    enumsspb.TASK_TYPE_REPLICATION_SYNC_HSM,
				TaskId:      task.TaskID,
			})
		default:
			panic(fmt.Sprintf("Unknown repication task type: %v", task))
		}
	}

	if len(taskInfo) == 0 {
		return nil, nil, ackLevel, pageToken, nil
	}

	dlqResponse, err := remoteAdminClient.GetDLQReplicationMessages(
		ctx,
		&adminservice.GetDLQReplicationMessagesRequest{
			TaskInfos: taskInfo,
		},
	)
	if err != nil {
		return nil, nil, ackLevel, nil, err
	}

	return dlqResponse.ReplicationTasks, taskInfo, ackLevel, pageToken, nil

View on GitHub (pinned to bde624efd1)

Solutions

1. Upgrade both replication clusters to matching temporal-server versions so all DLQ task types are known
2. Add a switch case for the missing task type (map it to its enumsspb TASK_TYPE_* and required fields), then regenerate any affected code
3. Inspect the DLQ to identify the offending task type printed in the panic
4. As a workaround, drain/delete the incompatible DLQ messages only if replication state allows it

Example fix

// before
default:
  panic(fmt.Sprintf("Unknown repication task type: %v", task))
// after
case *tasks.NewReplicationTaskType:
  taskInfo = append(taskInfo, &replicationspb.ReplicationTaskInfo{
    NamespaceId: task.NamespaceID, WorkflowId: task.WorkflowID, RunId: task.RunID,
    TaskType: enumsspb.TASK_TYPE_REPLICATION_NEW_TYPE, TaskId: task.TaskID,
  })
default:
  panic(fmt.Sprintf("Unknown repication task type: %v", task))

// Before reading DLQ messages, check task types are supported by this build:
for _, t := range resp.Tasks {
  switch t.(type) {
  case *tasks.SyncActivityTask, *tasks.HistoryReplicationTask,
    *tasks.SyncWorkflowStateTask, *tasks.SyncHSMTask:
  default:
    return fmt.Errorf("unsupported DLQ task type %T; upgrade clusters to matching versions", t)
  }
}

func isSupportedDLQTask(t tasks.Task) bool {
  switch t.(type) {
  case *tasks.SyncActivityTask, *tasks.HistoryReplicationTask,
    *tasks.SyncWorkflowStateTask, *tasks.SyncHSMTask:
    return true
  }
  return false
}

func safeGetMessages(h *replication.DLQHandler, req interface{}) (msgs interface{}, err error) {
  defer func() { if r := recover(); r != nil { err = fmt.Errorf("dlq read panic: %v", r) } }()
  return h.GetMessages(req)
}

When it happens

Trigger: A DLQ containing a replication task type not covered by the switch — typically after a temporal-server upgrade introduces a new task type (e.g. a newer sync task) while the DLQ handler code was not updated, or DLQ data written by a newer/older cluster version being read by this version.

Common situations: Multi-cluster replication with version skew between clusters; applying schema/code updates to one cluster before the other; custom forks adding new replication task types.

Related errors

• cannot handle replication task of type %v
• Failed to initialize replication DLQ handler due to nil task
• queue already exists
• BatchSize too large
• %w: state namespace failover version < ref namespace failove

AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/8d475f378df24e11. Report an issue: GitHub.