{"record":{"id":"cce3bd62682e25d0","repo":"abhigyanpatwari/GitNexus","slug":"worker-i-hit-a-deterministic-startup-crash-loop","errorCode":null,"errorMessage":"Worker ${i} hit a deterministic startup crash-loop; dropping slot without further retries.","messagePattern":"Worker (.+?) hit a deterministic startup crash-loop; dropping slot without further retries\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"gitnexus/src/core/ingestion/workers/worker-pool.ts","lineNumber":1291,"sourceCode":"        const msg = err instanceof Error ? err.message : String(err);\n        const sig = crashSignature(msg);\n        // Same signature as this slot's previous attempt => it survived a\n        // respawn, so retrying this slot is futile. (First crash has no prior\n        // signature, so attempt 0 never counts — every slot self-heals once.)\n        if (lastStartupSignature.get(i) === sig) reproducedStartupSlots.add(i);\n        lastStartupSignature.set(i, sig);\n        if (!anyWorkerReachedReady && reproducedStartupSlots.size >= deterministicSlotThreshold) {\n          deterministicStartupDetected = true;\n        }\n        await worker.terminate().catch(() => undefined);\n        workers[i] = undefined;\n\n        const giveUp =\n          terminated || deterministicStartupDetected || attempt >= STARTUP_RESTART_BUDGET;\n        if (giveUp) {\n          initialReadinessFailures.push(msg);\n          activeSlots.delete(i);\n          logger.warn(\n            { workerIndex: i, attempt, err: msg, deterministic: deterministicStartupDetected },\n            deterministicStartupDetected\n              ? `Worker ${i} hit a deterministic startup crash-loop; dropping slot without further retries.`\n              : `Worker ${i} did not report ready after ${attempt + 1} attempt(s); dropping slot.`,\n          );\n          return;\n        }\n        // Transient: jittered backoff, then respawn the slot and retry.\n        await abortableSleep(\n          startupBackoffMs(attempt),\n          () => terminated || deterministicStartupDetected,\n          pendingStartupTimers,\n        );\n        if (terminated || deterministicStartupDetected) {\n          initialReadinessFailures.push(msg);\n          activeSlots.delete(i);\n          return;\n        }","sourceCodeStart":1273,"sourceCodeEnd":1309,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/0d1aed942f0e8b5d3bac27519fff441aceea722d/gitnexus/src/core/ingestion/workers/worker-pool.ts#L1273-L1309","documentation":"Parsing workers must post a ready message after spawn. When a worker dies before ready, the pool respawns the slot with jittered backoff, up to STARTUP_RESTART_BUDGET (2) attempts, while tracking each crash's startup signature. If the same signature reproduces across enough slots (deterministicSlotThreshold) before any worker ever reached ready, the pool classifies the failure as deterministic — every respawn would fail identically — and drops the slot(s) with this warning instead of looping forever. The pool continues with the surviving workers.","triggerScenarios":"Something that crashes every worker identically at boot: a missing or ABI-incompatible native tree-sitter binding (Node upgraded without rebuilding node_modules), a corrupted or partial npm install, a broken worker bundle path, or container limits OOM-killing workers while grammars load.","commonSituations":"Upgrading Node.js without reinstalling gitnexus deps (NODE_MODULE_VERSION mismatch); interrupted installs leaving half-written native .node files; Docker/CI with tight memory or max-process limits; antivirus quarantining native binaries.","solutions":["Read the { err } (msg) field — it carries the crash reason (e.g. 'was compiled against a different Node.js version')","Rebuild dependencies for the current runtime: rm -rf node_modules && npm install in gitnexus/ (postinstall re-materializes vendored grammars; a matching platform-arch prebuild is preferred, else a C/C++ toolchain source-builds them)","Pin or verify a supported Node.js version and reinstall after any major upgrade","Raise container memory/process limits if msg shows OOM or spawn failures","Re-run analyze — with all slots dropped the run loses worker parallelism, so fixing the environment is required"],"exampleFix":"# before: Node upgraded, native tree-sitter bindings have stale ABI, every worker crash-loops\nnode -v && npx gitnexus analyze  # -> deterministic startup crash-loop\n\n# after: rebuild deps for the current Node, then re-index\ncd gitnexus && rm -rf node_modules && npm install\ncd .. && node .gitnexus/run.cjs analyze --index-only","handlingStrategy":"retry","validationCode":"// preflight in the host: load native bindings the workers will load\ntry {\n  require('web-tree-sitter');\n  for (const g of ['tree-sitter-c', 'tree-sitter-kotlin', 'tree-sitter-swift']) require(g);\n} catch (e) {\n  failFast(`native grammar load failed - workers will crash-loop: ${e.message}`);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["After any Node.js major upgrade, rm -rf node_modules && npm install in gitnexus/ so native tree-sitter bindings match the runtime ABI","Verify a platform-arch prebuild exists (or a python3/make/g++ toolchain for the source-build fallback) before deploying","Give containers enough memory and process slots for N workers plus the host","Read the { err } msg in the warn — 'compiled against a different Node.js version' means ABI mismatch, not a GitNexus bug"],"tags":["worker-threads","startup","crash-loop","native-modules","node-abi"],"backgroundTag":"worker-crash-loop","analyzedSha":"0d1aed942f0e8b5d3bac27519fff441aceea722d","analyzedAt":"2026-08-20T23:29:22.980Z","contentChangedAt":"2026-08-20T23:29:22.980Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}