temporalio/temporal · error

Task key Sub encountered underflow: self: %v, subtrahend: %v

Error message

Task key Sub encountered underflow: self: %v, subtrahend: %v

What it means

Key.Sub subtracts one key from another to yield the distance (e.g. for backlog metrics or lag computation). After borrowing across the TaskID boundary, if the resulting fireTime is still less than the subtrahend's fireTime, the subtraction underflows and the method panics — meaning 'self' is not greater than or equal to 'subtrahend'.

Source

Thrown at service/history/tasks/key.go:102

			panic("Key encountered positive overflow")
		}
		return NewKey(k.FireTime.Add(time.Nanosecond), 0)
	}
	return NewKey(k.FireTime, k.TaskID+1)
}

func (k Key) Sub(subtrahend Key) Key {
	borrow := int64(0)
	differenceTaskID := k.TaskID - subtrahend.TaskID
	if differenceTaskID < 0 {
		borrow = 1
		differenceTaskID += MaximumKey.TaskID
	}

	fireTime := k.FireTime.UnixNano() - borrow
	subtrahendFireTime := subtrahend.FireTime.UnixNano()
	if fireTime < subtrahendFireTime {
		panic(fmt.Sprintf("Task key Sub encountered underflow: self: %v, subtrahend: %v", k, subtrahend))
	}

	return NewKey(
		time.Unix(0, fireTime-subtrahendFireTime).UTC(),
		int64(differenceTaskID),
	)
}

func MinKey(this Key, that Key) Key {
	if this.CompareTo(that) < 0 {
		return this
	}
	return that
}

func MaxKey(this Key, that Key) Key {
	if this.CompareTo(that) < 0 {
		return that

View on GitHub (pinned to bde624efd1)

Solutions

1. Check argument order: compute largerKey.Sub(smallerKey); verify self >= subtrahend before calling.
2. Guard the call site with a comparison and clamp the result to zero when self <= subtrahend.
3. If clocks jumped backwards (NTP correction), reset the affected cursor/task-min-scheduled-time so keys stay monotonic.
4. Log both keys at the call site to identify which component produced the out-of-order keys.

Example fix

// before
lag := maxKey.Sub(minKey) // panics if minKey >= maxKey

// after
var lag tasks.Key
if maxKey.CompareTo(minKey) > 0 {
    lag = maxKey.Sub(minKey)
}

if self.CompareTo(subtrahend) <= 0 {
    return 0 // no lag; skip Sub
}

When it happens

Trigger: Calling keyA.Sub(keyB) where keyB >= keyA, e.g. computing lag with a cursor that has moved past the comparison key, or passing arguments in the wrong order (min.Sub(max) instead of max.Sub(min)).

Common situations: Backlog/standby lag metric loops after clocks change or cursors reset; comparing keys from different shards or clusters with unsynchronized clocks; refactors that swap the argument order.

Related errors

• Key encountered negative underflow
• Key encountered positive overflow
• Unknown category type: %v
• Found key with non-zero pending task count but has no corres
• unknown task predicate type: %T

AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/101e1786b6ca7dd6. Report an issue: GitHub.