temporalio/temporal · error

BatchSize too large

Error message

BatchSize too large

What it means

ErrMergeBatchSizeTooLarge is declared in the DLQ merge workflow and returned by parseMergeParams when the user-supplied BatchSize for a DLQ merge operation exceeds the allowed maximum. Merging DLQ messages processes each batch as one unit, so an oversized batch would put excessive load on persistence; the workflow refuses to start instead.

Source

Thrown at service/worker/dlq/workflow.go:174

	deleteTasksActivityName      = "dlq-delete-tasks-activity"
	readTasksActivityName        = "dlq-read-tasks-activity"
	reEnqueueTasksActivityName   = "dlq-re-enqueue-tasks-activity"

	// deleteTasksActivityTimeout is long because all tasks are deleted in a single go. This only applies when using the
	// purge workflow, not when the delete activity is used in the merge workflow.
	deleteTasksActivityTimeout = 5 * time.Minute * debug.TimeoutMultiplier
	// mergeTasksActivityTimeout controls the timeout of all activities used in the merge workflow. It is relatively
	// short because we're only processing a single batch of tasks at a time.
	mergeTasksActivityTimeout = 15 * time.Second * debug.TimeoutMultiplier
)

var (
	// Module provides a [workercommon.WorkerComponent] annotated with [workercommon.WorkerComponentTag] to the graph,
	// given a [HistoryClient], a [TaskClientDialer], and a value for [CurrentClusterName].
	Module = workercommon.AnnotateWorkerComponentProvider(newComponent)

	ErrNegativeBatchSize      = errors.New("BatchSize must be positive or 0 to use the default")
	ErrMergeBatchSizeTooLarge = errors.New("BatchSize too large")

	// deleteActivityRetryPolicy is the retry policy for the delete activity. Currently, delete processes all messages
	// in one batch, so this could be expensive. As a result, we want to increase the backoff quickly to not peg the
	// system.
	deleteActivityRetryPolicy = &temporal.RetryPolicy{
		InitialInterval:    100 * time.Millisecond,
		BackoffCoefficient: 2.0,
		MaximumAttempts:    10,
	}
	// mergeActivityRetryPolicy is the retry policy for the merge activities. Currently, merge processes one batch of
	// messages at a time, and each batch has a capped size determined by MaxMergeBatchSize, so this is relatively
	// cheap. As a result, we want to increase the backoff slowly because it's unlikely that this will hurt the system.
	mergeActivityRetryPolicy = &temporal.RetryPolicy{
		InitialInterval:    100 * time.Millisecond,
		BackoffCoefficient: 1.2,
		MaximumAttempts:    10,
	}
)

View on GitHub (pinned to bde624efd1)

Solutions

1. Lower the BatchSize in MergeDLQMessagesParams to a value within the allowed maximum (or 0 to use the default).
2. Run merges without specifying BatchSize so the default is used.
3. Check the DLQ workflow's configured max batch size constant/config in dlq/workflow.go and align tooling with it.
4. Drain a large DLQ with multiple sequential merge operations instead of one oversized batch.

Example fix

// before
params := &workerdldb.MergeDLQMessagesParams{TargetCluster: "active", BatchSize: 1000000}
// after
params := &workerdldb.MergeDLQMessagesParams{TargetCluster: "active", BatchSize: 100} // within limit, or 0 for default

const maxMergeBatchSize = 1000 // check dlq/workflow.go for the actual limit
if params.BatchSize > maxMergeBatchSize {
	return fmt.Errorf("BatchSize %d exceeds max %d; use 0 for default", params.BatchSize, maxMergeBatchSize)
}

func validMergeBatchSize(n int) bool { return n >= 0 } // 0 = default; enforce upper bound before calling

err := dlqWorker.MergeDLQMessages(ctx, params)
if errors.Is(err, dlq.ErrMergeBatchSizeTooLarge) {
	params.BatchSize = 0 // fall back to default size and retry
}

When it happens

Trigger: Calling the DLQ merge workflow (e.g. `temporal dlq merge` or MergeDLQMessages workflow) with a MergeDLQMessagesParams whose BatchSize exceeds the configured maximum batch size.

Common situations: Operators trying to speed up DLQ draining by setting a huge batch size; copy-pasting a default that no longer matches a lowered server-side limit after an upgrade or config change.

Related errors

• queue already exists
• batchParams is nil
• out of range
• conflicting timezone names
• ${errs}

AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/3cc606833deae734. Report an issue: GitHub.