Queue slice encountered task doesn't belong to either scopes

Error message

Queue slice encountered task doesn't belong to either scopes during split, scope: %v and %v, task: %v, task type: %v

What it means

When an executableTracker splits, every pending executable must belong to exactly one of the two resulting scopes (thisScope or thatScope). This panic fires when a pending task falls into neither, meaning the split scopes have a gap or the task predates a scope change. It is an internal invariant protecting task ownership during queue splits.

Source

Thrown at service/history/queues/tracker.go:44

}

func (t *executableTracker) split(
	thisScope Scope,
	thatScope Scope,
) (*executableTracker, *executableTracker) {
	that := executableTracker{
		pendingExecutables: make(map[tasks.Key]Executable, len(t.pendingExecutables)/2),
		grouper:            t.grouper,
		pendingPerKey:      make(map[any]int, len(t.pendingPerKey)),
	}

	for key, executable := range t.pendingExecutables {
		if thisScope.Contains(executable) {
			continue
		}

		if !thatScope.Contains(executable) {
			panic(fmt.Sprintf("Queue slice encountered task doesn't belong to either scopes during split, scope: %v and %v, task: %v, task type: %v",
				thisScope, thatScope, executable.GetTask(), executable.GetType()))
		}

		delete(t.pendingExecutables, key)
		that.pendingExecutables[key] = executable

		groupKey := t.grouper.Key(executable)
		t.pendingPerKey[groupKey]--
		that.pendingPerKey[groupKey]++
	}

	return t, &that
}

func (t *executableTracker) merge(incomingTracker *executableTracker) *executableTracker {
	thisExecutables, thisPendingTasks := t.pendingExecutables, t.pendingPerKey
	thatExecutables, thatPendingTasks := incomingTracker.pendingExecutables, incomingTracker.pendingPerKey
	if len(thisExecutables) < len(thatExecutables) {

View on GitHub (pinned to bde624efd1)

Solutions

Ensure scopes produced by a split exactly partition the original scope (no gaps)
Verify tasks cannot be added to the tracker after their slice's scope was shrunk
Check that executable.GetType()/GetTask() in the panic message to identify the producer, then trace why its key escaped both scopes
Report to temporalio/temporal with both scopes and the task details if reproducible

Example fix

// before: shrinking scope without clearing tasks loaded under the old scope
slice.ShrinkScope() // pending executables may now fall outside both split scopes
// after: clear or re-bucket pending tasks when narrowing the scope (as Clear does)
slice.Clear()

Defensive patterns

Strategy: validation

Validate before calling

// Before splitting, verify the two new scopes partition the old scope:
if !oldScope.Range.Equal(newThisScope.Range.Union(newThatScope.Range)) {
  return fmt.Errorf("split scopes do not cover original range")
}

Try / catch

func safeSplit(t queues.ExecutableTracker, a, b queues.Scope) (err error) {
  defer func() { if r := recover(); r != nil { err = fmt.Errorf("split panic: %v", r) } }()
  t.Split(a, b)
  return nil
}

Prevention

Ensure split scopes exactly partition the original scope with no gaps
Clear or re-bucket pending tasks before shrinking a slice's scope
Avoid mutating scope while tasks are being loaded via SelectTasks

When it happens

Trigger: splitByRange or SplitByPredicate running while pendingExecutables contains a task outside both new scopes — e.g. scope shrunk (ShrinkScope) after tasks were loaded, or split ranges/predicate computed inconsistently with what was loaded earlier.

Common situations: Seen in temporal-server history service during aggressive queue splitting/merging under memory pressure; usually a bug in range or predicate computation, or a task added outside SelectTasks' range check; forked iterators or upgrade skew can also trigger it.

Related errors

AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/941abe8c130a5164. Report an issue: GitHub.