What causes "BUG: thread with ID: %v already finished" in vitessio/vitess?

Return from the worker loop immediately after calling ThreadFinished()

BUG: thread with ID: %v already finished

Error message

BUG: thread with ID: %v already finished

What it means

ThrottlerImpl.Throttle() panics if the given threadID was already marked finished via ThreadFinished(). The throttler's contract is that after ThreadFinished() returns, that thread must never call Throttle() again; violating it indicates a worker loop that kept running past its completion signal.

Source

Thrown at go/vt/throttler/throttler.go:237

		}
	}()

	return t, nil
}

// Throttle returns a backoff duration which specifies for how long "threadId"
// should wait before it issues the next request.
// If the duration is zero, the thread is not throttled.
// If the duration is not zero, the thread must call Throttle() again after
// the backoff duration elapsed.
// The maximum value for the returned backoff is 1 second since the throttler
// internally operates on a per-second basis.
func (t *ThrottlerImpl) Throttle(threadID int) time.Duration {
	if t.closed {
		panic(fmt.Sprintf("BUG: thread with ID: %v must not access closed Throttler", threadID))
	}
	if t.threadFinished[threadID] {
		panic(fmt.Sprintf("BUG: thread with ID: %v already finished", threadID))
	}
	return t.threadThrottlers[threadID].throttle(t.nowFunc())
}

// MaxLag returns the max of all the last replication lag values seen across all tablets of
// the provided type, excluding ignored tablets.
func (t *ThrottlerImpl) MaxLag(tabletType topodata.TabletType) uint32 {
	cache := t.maxReplicationLagModule.lagCacheByType(tabletType)
	if cache == nil {
		return 0
	}
	return cache.maxLag()
}

// ThreadFinished marks threadID as finished and redistributes the thread's
// rate allotment across the other threads.
// After ThreadFinished() is called, Throttle() must not be called anymore.
func (t *ThrottlerImpl) ThreadFinished(threadID int) {

View on GitHub (pinned to 01a25a7d17)

Solutions

Return from the worker loop immediately after calling ThreadFinished()
Ensure every thread registers a unique threadID (check newThread() usage)
Add a done-channel check before each Throttle call in the worker loop

Example fix

// before
throttler.ThreadFinished(threadID)
for { throttler.Throttle(threadID) }
// after
throttler.ThreadFinished(threadID)
return

Defensive patterns

Strategy: validation

Validate before calling

if finished[threadID] {
    return // skip Throttle; thread already marked finished
}
d := throttler.Throttle(threadID)

Try / catch

defer func() {
    if r := recover(); r != nil {
        if strings.Contains(fmt.Sprint(r), "already finished") {
            return
        }
        panic(r)
    }
}()

Prevention

Return immediately after ThreadFinished()
Ensure unique threadID per goroutine
Check a stop flag before each Throttle iteration

When it happens

Trigger: Calling Throttle(threadID) for a threadID on which ThreadFinished(threadID) was already called — typically a worker goroutine that continues its loop after marking itself finished, or a duplicated/reused threadID.

Common situations: Worker goroutines not respecting a stop flag before their next Throttle iteration; registering two threads with the same ID so one's ThreadFinished marks the other finished prematurely.

Related errors

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/27c178369e67a418. Report an issue: GitHub.