abhigyanpatwari/GitNexus · warning

Worker ${workerIndex} exceeded respawn budget; dropping slot

Error message

Worker ${workerIndex} exceeded respawn budget; dropping slot.

What it means

After a worker death, each slot counts respawns; when respawnCount exceeds maxRespawnsPerSlot (option, or GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT), the slot is dropped: the worker is removed and the slot leaves activeSlots. If it was the last active slot, the pool trips the breaker with WorkerPoolDispatchError citing exhaustion; otherwise dispatch continues on the remaining slots.

Source

Thrown at gitnexus/src/core/ingestion/workers/worker-pool.ts:1783

        if (stopped) return;
        consecutiveFailuresPerSlot[workerIndex]++;
        for (const p of excludePaths) {
          if (p) quarantine.add(p);
        }
        if (consecutiveFailuresPerSlot[workerIndex] >= poolOptions.consecutiveFailureThreshold) {
          tripBreaker(
            new WorkerPoolDispatchError(
              `${reason}. Pool circuit breaker tripped: slot ${workerIndex} hit ` +
                `${consecutiveFailuresPerSlot[workerIndex]} consecutive failures ` +
                `(threshold: ${poolOptions.consecutiveFailureThreshold}).`,
              quarantine.snapshot(),
            ),
          );
          return;
        }
        respawnCount[workerIndex]++;
        if (respawnCount[workerIndex] > poolOptions.maxRespawnsPerSlot) {
          logger.warn(
            {
              workerIndex,
              respawnCount: respawnCount[workerIndex],
              maxRespawns: poolOptions.maxRespawnsPerSlot,
              reason,
            },
            `Worker ${workerIndex} exceeded respawn budget; dropping slot.`,
          );
          await removeWorkerFromSlot(workerIndex, removalMode, reason);
          activeSlots.delete(workerIndex);
          if (activeSlots.size === 0) {
            tripBreaker(
              new WorkerPoolDispatchError(
                `${reason}. All ${size} worker slot(s) exhausted their respawn budget.`,
                quarantine.snapshot(),
              ),
            );
            return;

View on GitHub (pinned to 0d1aed942f)

Solutions

  1. Read the reason field: heap-cap deaths say to raise GITNEXUS_WORKER_HEAP_MB; native/parse deaths point to a specific input or grammar
  2. Fix the crash source (exclude the pathological file, fix the native install) — respawns treat symptoms, not causes
  3. Raise GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT only for genuinely transient environments, paired with lower parallelism
  4. If dispatch later fails with 'All N worker slot(s) exhausted their respawn budget', treat that error's quarantine snapshot as the culprit list

Example fix

# before
export GITNEXUS_WORKER_POOL_SIZE=8   # repeated heap-cap deaths burn respawn budget → slot dropped
# after
export GITNEXUS_WORKER_POOL_SIZE=4
export GITNEXUS_WORKER_HEAP_MB=1024  # fewer deaths, budget preserved
Defensive patterns

Strategy: validation

Validate before calling

// Pre-flight respawn economics: transient-death-heavy environments need headroom.
const budget = Number(process.env.GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT ?? 0);
if (budget < expectedCrashesPerSlot) {
  process.env.GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT = String(expectedCrashesPerSlot);
}

Prevention

When it happens

Trigger: handleWorkerDeath incrementing respawnCount past the per-slot budget through repeated worker exits — native crashes, OOM deaths at the heap cap, or abort-prone files killing each replacement soon after spawn; the warn fires with workerIndex, respawnCount, maxRespawns, and reason.

Common situations: A repo containing several parser-crash files so every respawn dies again quickly; memory-constrained runners where each worker OOMs (heap cap) within minutes; or an ABI-broken native grammar crashing all workers deterministically.

Related errors


AI-assisted analysis of abhigyanpatwari/GitNexus@0d1aed942f (2026-08-20). Data as JSON: /api/errors/0ac5772b9efaa7f3. Report an issue: GitHub.