abhigyanpatwari/GitNexus · warning
Worker ${workerIndex} died; respawning slot (attempt ${respa
Error message
Worker ${workerIndex} died; respawning slot (attempt ${respawnCount[workerIndex]}/${poolOptions.maxRespawnsPerSlot}). What it means
The per-death counterpart of the respawn-budget warn: whenever a worker dies and both the per-slot consecutive-failure breaker and the respawn budget still have headroom, the slot logs this warn (workerIndex, respawnCount/maxRespawns, reason, excludePaths) and calls replaceWorker to respawn it. Dispatch continues on the new worker.
Source
Thrown at gitnexus/src/core/ingestion/workers/worker-pool.ts:1805
maxRespawns: poolOptions.maxRespawnsPerSlot,
reason,
},
`Worker ${workerIndex} exceeded respawn budget; dropping slot.`,
);
await removeWorkerFromSlot(workerIndex, removalMode, reason);
activeSlots.delete(workerIndex);
if (activeSlots.size === 0) {
tripBreaker(
new WorkerPoolDispatchError(
`${reason}. All ${size} worker slot(s) exhausted their respawn budget.`,
quarantine.snapshot(),
),
);
return;
}
return;
}
logger.warn(
{
workerIndex,
respawnCount: respawnCount[workerIndex],
reason,
excludePaths,
},
`Worker ${workerIndex} died; respawning slot (attempt ${respawnCount[workerIndex]}/${poolOptions.maxRespawnsPerSlot}).`,
);
const respawned = await replaceWorker(workerIndex, removalMode, reason);
if (!respawned) {
activeSlots.delete(workerIndex);
if (activeSlots.size === 0) {
tripBreaker(
new WorkerPoolDispatchError(
`${reason}. Replacement worker startup failed and no slots remain.`,
quarantine.snapshot(),
),
);View on GitHub (pinned to 0d1aed942f)
Solutions
- Treat a single occurrence as normal self-healing; correlate the reason field with the job/file being parsed at death time
- If repeats cluster on a file, expect quarantine (unattributable) or exclusion (attributed) to stop the bleeding — then address that file
- For heap-cap reasons, raise GITNEXUS_WORKER_HEAP_MB or reduce GITNEXUS_WORKER_POOL_SIZE to stop systematic OOM
- Track respawnCount in the warns: approaching maxRespawnsPerSlot means the root cause must be fixed before slots start dropping
Example fix
# before export GITNEXUS_WORKER_HEAP_MB=256 # repeated heap-cap deaths → constant respawn warns # after export GITNEXUS_WORKER_HEAP_MB=1024 # deaths stop, respawn warns stop
Defensive patterns
Strategy: retry
Validate before calling
// Watch respawn pressure and intervene before slots drop:
const warns = parseWorkerWarns(logText).filter((w) => w.msg.startsWith('Worker ') && w.msg.includes('died; respawning'));
const nearBudget = warns.filter((w) => w.respawnCount >= w.maxRespawns - 1);
if (nearBudget.length > 0) failFastWithReasons(nearBudget.map((w) => w.reason)); Prevention
- Correlate each death's reason with the file being parsed at that moment
- Fix systematic OOM with GITNEXUS_WORKER_HEAP_MB / smaller pools instead of tolerating respawn churn
- Track respawnCount trends across runs — growth always precedes slot exhaustion
When it happens
Trigger: Any worker exit event (crash, kill, native abort avoided earlier) that reaches the respawn branch: consecutiveFailuresPerSlot is under the threshold, respawnCount is at or below maxRespawnsPerSlot, so the slot is replaced and the attempt counter is announced.
Common situations: Occasional worker OOMs on huge files, sporadic native grammar crashes, CI runners with memory spikes — each death produces one warn and a fresh worker; frequent repeats escalate to the budget-exceeded warn and eventually the breaker.
Related errors
- Request failed after retries (HTTP ${response.status})
- Worker ${i} crashed during startup; respawning slot (self-he
- Conceptual job ${job.startIndex} died ${deaths} times unattr
- Conceptual job ${job.startIndex} died ${deaths} times unattr
- Worker ${workerIndex} exceeded respawn budget; dropping slot
AI-assisted analysis of abhigyanpatwari/GitNexus@0d1aed942f (2026-08-20).
Data as JSON: /api/errors/d396b0f4da5c2023.
Report an issue: GitHub.