abhigyanpatwari/GitNexus · warning

Worker ${workerIndex} died; respawning slot (attempt ${respa

Error message

Worker ${workerIndex} died; respawning slot (attempt ${respawnCount[workerIndex]}/${poolOptions.maxRespawnsPerSlot}).

What it means

The per-death counterpart of the respawn-budget warn: whenever a worker dies and both the per-slot consecutive-failure breaker and the respawn budget still have headroom, the slot logs this warn (workerIndex, respawnCount/maxRespawns, reason, excludePaths) and calls replaceWorker to respawn it. Dispatch continues on the new worker.

Source

Thrown at gitnexus/src/core/ingestion/workers/worker-pool.ts:1805

              maxRespawns: poolOptions.maxRespawnsPerSlot,
              reason,
            },
            `Worker ${workerIndex} exceeded respawn budget; dropping slot.`,
          );
          await removeWorkerFromSlot(workerIndex, removalMode, reason);
          activeSlots.delete(workerIndex);
          if (activeSlots.size === 0) {
            tripBreaker(
              new WorkerPoolDispatchError(
                `${reason}. All ${size} worker slot(s) exhausted their respawn budget.`,
                quarantine.snapshot(),
              ),
            );
            return;
          }
          return;
        }
        logger.warn(
          {
            workerIndex,
            respawnCount: respawnCount[workerIndex],
            reason,
            excludePaths,
          },
          `Worker ${workerIndex} died; respawning slot (attempt ${respawnCount[workerIndex]}/${poolOptions.maxRespawnsPerSlot}).`,
        );
        const respawned = await replaceWorker(workerIndex, removalMode, reason);
        if (!respawned) {
          activeSlots.delete(workerIndex);
          if (activeSlots.size === 0) {
            tripBreaker(
              new WorkerPoolDispatchError(
                `${reason}. Replacement worker startup failed and no slots remain.`,
                quarantine.snapshot(),
              ),
            );

View on GitHub (pinned to 0d1aed942f)

Solutions

  1. Treat a single occurrence as normal self-healing; correlate the reason field with the job/file being parsed at death time
  2. If repeats cluster on a file, expect quarantine (unattributable) or exclusion (attributed) to stop the bleeding — then address that file
  3. For heap-cap reasons, raise GITNEXUS_WORKER_HEAP_MB or reduce GITNEXUS_WORKER_POOL_SIZE to stop systematic OOM
  4. Track respawnCount in the warns: approaching maxRespawnsPerSlot means the root cause must be fixed before slots start dropping

Example fix

# before
export GITNEXUS_WORKER_HEAP_MB=256   # repeated heap-cap deaths → constant respawn warns
# after
export GITNEXUS_WORKER_HEAP_MB=1024  # deaths stop, respawn warns stop
Defensive patterns

Strategy: retry

Validate before calling

// Watch respawn pressure and intervene before slots drop:
const warns = parseWorkerWarns(logText).filter((w) => w.msg.startsWith('Worker ') && w.msg.includes('died; respawning'));
const nearBudget = warns.filter((w) => w.respawnCount >= w.maxRespawns - 1);
if (nearBudget.length > 0) failFastWithReasons(nearBudget.map((w) => w.reason));

Prevention

When it happens

Trigger: Any worker exit event (crash, kill, native abort avoided earlier) that reaches the respawn branch: consecutiveFailuresPerSlot is under the threshold, respawnCount is at or below maxRespawnsPerSlot, so the slot is replaced and the attempt counter is announced.

Common situations: Occasional worker OOMs on huge files, sporadic native grammar crashes, CI runners with memory spikes — each death produces one warn and a fresh worker; frequent repeats escalate to the budget-exceeded warn and eventually the breaker.

Related errors


AI-assisted analysis of abhigyanpatwari/GitNexus@0d1aed942f (2026-08-20). Data as JSON: /api/errors/d396b0f4da5c2023. Report an issue: GitHub.