abhigyanpatwari/GitNexus · warning
Worker ${workerIndex} exceeded respawn budget; dropping slot
Error message
Worker ${workerIndex} exceeded respawn budget; dropping slot. What it means
After a worker death, each slot counts respawns; when respawnCount exceeds maxRespawnsPerSlot (option, or GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT), the slot is dropped: the worker is removed and the slot leaves activeSlots. If it was the last active slot, the pool trips the breaker with WorkerPoolDispatchError citing exhaustion; otherwise dispatch continues on the remaining slots.
Source
Thrown at gitnexus/src/core/ingestion/workers/worker-pool.ts:1783
if (stopped) return;
consecutiveFailuresPerSlot[workerIndex]++;
for (const p of excludePaths) {
if (p) quarantine.add(p);
}
if (consecutiveFailuresPerSlot[workerIndex] >= poolOptions.consecutiveFailureThreshold) {
tripBreaker(
new WorkerPoolDispatchError(
`${reason}. Pool circuit breaker tripped: slot ${workerIndex} hit ` +
`${consecutiveFailuresPerSlot[workerIndex]} consecutive failures ` +
`(threshold: ${poolOptions.consecutiveFailureThreshold}).`,
quarantine.snapshot(),
),
);
return;
}
respawnCount[workerIndex]++;
if (respawnCount[workerIndex] > poolOptions.maxRespawnsPerSlot) {
logger.warn(
{
workerIndex,
respawnCount: respawnCount[workerIndex],
maxRespawns: poolOptions.maxRespawnsPerSlot,
reason,
},
`Worker ${workerIndex} exceeded respawn budget; dropping slot.`,
);
await removeWorkerFromSlot(workerIndex, removalMode, reason);
activeSlots.delete(workerIndex);
if (activeSlots.size === 0) {
tripBreaker(
new WorkerPoolDispatchError(
`${reason}. All ${size} worker slot(s) exhausted their respawn budget.`,
quarantine.snapshot(),
),
);
return;View on GitHub (pinned to 0d1aed942f)
Solutions
- Read the reason field: heap-cap deaths say to raise GITNEXUS_WORKER_HEAP_MB; native/parse deaths point to a specific input or grammar
- Fix the crash source (exclude the pathological file, fix the native install) — respawns treat symptoms, not causes
- Raise GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT only for genuinely transient environments, paired with lower parallelism
- If dispatch later fails with 'All N worker slot(s) exhausted their respawn budget', treat that error's quarantine snapshot as the culprit list
Example fix
# before export GITNEXUS_WORKER_POOL_SIZE=8 # repeated heap-cap deaths burn respawn budget → slot dropped # after export GITNEXUS_WORKER_POOL_SIZE=4 export GITNEXUS_WORKER_HEAP_MB=1024 # fewer deaths, budget preserved
Defensive patterns
Strategy: validation
Validate before calling
// Pre-flight respawn economics: transient-death-heavy environments need headroom.
const budget = Number(process.env.GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT ?? 0);
if (budget < expectedCrashesPerSlot) {
process.env.GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT = String(expectedCrashesPerSlot);
} Prevention
- Fix crash sources (heap caps, native grammar ABI, poison files) before raising respawn budgets
- Read the reason field on every respawn warn — heap-cap reasons point at memory, not luck
- When the last slot drops, the WorkerPoolDispatchError carries the quarantine snapshot: use it for the post-mortem
When it happens
Trigger: handleWorkerDeath incrementing respawnCount past the per-slot budget through repeated worker exits — native crashes, OOM deaths at the heap cap, or abort-prone files killing each replacement soon after spawn; the warn fires with workerIndex, respawnCount, maxRespawns, and reason.
Common situations: A repo containing several parser-crash files so every respawn dies again quickly; memory-constrained runners where each worker OOMs (heap cap) within minutes; or an ABI-broken native grammar crashing all workers deterministically.
Related errors
- Worker ${i} crashed during startup; respawning slot (self-he
- Conceptual job ${job.startIndex} died ${deaths} times unattr
- Conceptual job ${job.startIndex} died ${deaths} times unattr
- Worker ${workerIndex} died; respawning slot (attempt ${respa
- Circuit '${key}' is open; retry in ${Math.ceil(retryAfterMs
AI-assisted analysis of abhigyanpatwari/GitNexus@0d1aed942f (2026-08-20).
Data as JSON: /api/errors/0ac5772b9efaa7f3.
Report an issue: GitHub.