Worker pool may overcommit memory: ${size} workers × ${worke

Error message

Worker pool may overcommit memory: ${size} workers × ${workerHeapCapMb}MB heap cap exceeds 60% of the ${effectiveMb}MB available to this process. Reduce GITNEXUS_WORKER_POOL_SIZE or set GITNEXUS_WORKER_HEAP_MB.

What it means

At pool construction the per-worker heap cap (512MB floor, or GITNEXUS_WORKER_HEAP_MB) times the pool size is compared against process-available RAM. If the committed total exceeds 60% of effective memory, the pool warns about potential overcommit (#2649 review): behavior is unchanged (deaths stay attributed, quarantine converges) but the operator is told up front instead of discovering it via worker OOMs.

Source

Thrown at gitnexus/src/core/ingestion/workers/worker-pool.ts:1083

  const parsedFileStoreStoragePath = options?.parsedFileStoreStoragePath;
  const durableParsedFileStoragePath = options?.durableParsedFileStoragePath;
  // CFG/PDG opt-in (#2081 M1) — carried in workerData alongside the store paths.
  const pdg = options?.pdg === true;
  const pdgMaxFunctionLines = options?.pdgMaxFunctionLines;
  const workerStoreData =
    parsedFileStoreStoragePath || durableParsedFileStoragePath || pdg
      ? { parsedFileStoreStoragePath, durableParsedFileStoragePath, pdg, pdgMaxFunctionLines }
      : undefined;
  const workerHeapCapMb = resolveWorkerHeapCapMb(size);
  // The 512MB per-worker floor exists so a worker can parse anything real,
  // but on a very small container a large pool of floored workers can still
  // overcommit total memory (#2649 review). Behavior is unchanged — deaths
  // are attributed and quarantine converges — but say so up front, with the
  // two levers, instead of letting the operator discover it from worker OOMs.
  const poolCommitMb = workerHeapCapMb * size;
  const effectiveMb = Math.floor(effectiveRamBytes() / (1024 * 1024));
  if (poolCommitMb > 0.6 * effectiveMb) {
    logger.warn(
      { poolSize: size, workerHeapCapMb, effectiveMb },
      `Worker pool may overcommit memory: ${size} workers × ${workerHeapCapMb}MB heap cap exceeds 60% of the ${effectiveMb}MB available to this process. Reduce GITNEXUS_WORKER_POOL_SIZE or set GITNEXUS_WORKER_HEAP_MB.`,
    );
  }
  // #2649 stall probe: test seam wins; production uses the heartbeat tracker.
  const stallTracker = options?.stallMsProbe
    ? { read: options.stallMsProbe, stop: (): void => undefined }
    : startHeartbeatStallTracker();
  const spawnWorker =
    options?.workerFactory ??
    ((url: URL) =>
      new Worker(url, {
        // Piped (not inherited) stdio: stderr for crash capture (#1741),
        // stdout because inherited stdout triggers silent startup crashes on
        // some hosts (see forwardWorkerStdout).
        stdout: true,
        stderr: true,
        workerData: workerStoreData,

View on GitHub (pinned to 0d1aed942f)

Solutions

Reduce GITNEXUS_WORKER_POOL_SIZE (or pass --workers N) so size × heap ≤ ~60% of available RAM
Set GITNEXUS_WORKER_HEAP_MB explicitly to size the per-worker cap for your container rather than relying on the 512MB floor
Run on/allocate more memory if the parallelism is genuinely needed for indexing throughput
Treat subsequent worker OOM deaths (heap-cap messages) as confirmation — raise memory or lower parallelism, not retries

Example fix

# before (2GB container)
export GITNEXUS_WORKER_POOL_SIZE=8   # 8 × 512MB = 4096MB > 60% of 2048MB → warn
# after
export GITNEXUS_WORKER_POOL_SIZE=2
export GITNEXUS_WORKER_HEAP_MB=512   # 1024MB ≤ 1228MB budget → no warn

Defensive patterns

Strategy: validation

Validate before calling

// Pre-flight the commit before constructing the pool:
const size = Number(process.env.GITNEXUS_WORKER_POOL_SIZE ?? defaultPoolSize());
const heapMb = Number(process.env.GITNEXUS_WORKER_HEAP_MB ?? 512);
const availableMb = Math.floor(os.totalmem() / 1024 / 1024); // or cgroup limit
if (size * heapMb > 0.6 * availableMb) {
  process.env.GITNEXUS_WORKER_POOL_SIZE = String(Math.max(1, Math.floor((0.6 * availableMb) / heapMb)));
}

Prevention

Size pools against the cgroup/container limit, not host RAM, in CI
Set GITNEXUS_WORKER_HEAP_MB explicitly instead of inheriting the 512MB floor blindly
Treat any later heap-cap worker death as a sizing bug, not a retry candidate

When it happens

Trigger: Creating the worker pool with size × workerHeapCapMb > 0.6 × effectiveMb — e.g. a large GITNEXUS_WORKER_POOL_SIZE on a small container, or the 512MB floor applying across many workers on a low-RAM box; effectiveRamBytes() determines the denominator.

Common situations: CI containers with 1–2GB RAM and default pool sizing, self-hosted runners where pool size was copied from a beefier machine, or GITNEXUS_WORKER_HEAP_MB raised without accounting for worker count.

Related errors

AI-assisted analysis of abhigyanpatwari/GitNexus@0d1aed942f (2026-08-20). Data as JSON: /api/errors/a20c72b1d66e568f. Report an issue: GitHub.