abhigyanpatwari/GitNexus · error · Error

Analyze stopped before running out of memory: ${heapUsedMB}M

Error message

Analyze stopped before running out of memory: ${heapUsedMB}MB of the ${heapLimitMB}MB Node heap in use at parse chunk ${chunkIdx + 1}/${numChunks} (#2649). ${heapPressureRemedy}

What it means

A mid-loop heap guard that aborts the parse phase BEFORE V8 enters its ineffective-mark-compact death spiral. It samples `process.memoryUsage().heapUsed` against `v8.getHeapStatistics().heap_size_limit` at each chunk boundary and throws when heap use exceeds 92% (`HEAP_ABORT_FRACTION`) of the limit. The throw is intentional: above ~95% V8 produces multi-second GC pauses that also falsely idle-timeout healthy workers. The worker pool is torn down by the enclosing `finally`. The abort can be declined by setting `GITNEXUS_MEMORY=off`.

Source

Thrown at gitnexus/src/core/ingestion/pipeline-phases/parse-impl.ts:971

        }
      }
      await applyChunkResults(chunkWorkerData, p.chunkIdx, p.chunkFiles, p.chunkStartMs);
    };

    for (let chunkIdx = 0; chunkIdx < numChunks; chunkIdx++) {
      if (heapProbeEveryN > 0 && chunkIdx > 0 && chunkIdx % heapProbeEveryN === 0) {
        logHeapProbe(
          `parse-chunk-${chunkIdx}`,
          `nodes=${graph.nodeCount} parsedFiles=${allParsedFiles.length}`,
        );
      }
      // #2649 mid-loop heap guard: fail actionably BEFORE V8 enters the
      // ineffective-mark-compact death spiral (which also falsely times out
      // healthy workers). The pool is torn down by this function's finally.
      const heapUsedNow = process.memoryUsage().heapUsed;
      const heapLimitNow = v8.getHeapStatistics().heap_size_limit;
      if (shouldAbortForHeapPressure(heapUsedNow, heapLimitNow)) {
        throw new Error(
          `Analyze stopped before running out of memory: ${Math.round(heapUsedNow / 1024 / 1024)}MB of the ` +
            `${Math.round(heapLimitNow / 1024 / 1024)}MB Node heap in use at parse chunk ${chunkIdx + 1}/${numChunks} (#2649). ` +
            heapPressureRemedy(heapLimitNow),
        );
      }
      const chunkPaths = chunks[chunkIdx];
      // Start wall-clock for the per-chunk throughput log emitted at end
      // of this iteration. The gate is computed once above; here we just
      // sample the clock if the gate is on. Computed when either
      // NODE_ENV=development OR the operator passed `--verbose`
      // (GITNEXUS_VERBOSE) — the previous `isDev`-only gate meant
      // operators running `gitnexus analyze --verbose` in production
      // never saw the log (M3 from PR #1693 review).
      const chunkStartMs: number | null = verboseThroughputLog ? Date.now() : null;

      const chunkContentPromise = chunkContentPromises[chunkIdx];
      if (!chunkContentPromise) {
        throw new Error(`Missing prefetched parse chunk ${chunkIdx + 1}/${numChunks}`);

View on GitHub (pinned to d540b00184)

Solutions

1. If the message says 'This machine has more memory available' — remove the `--max-old-space-size` pin from NODE_OPTIONS / node flags so the RAM-aware auto-sizer grants a larger heap.
2. If the message says 'at its memory ceiling' — exclude generated/vendored directories via a `.gitnexusignore` file to shrink the parseable set.
3. Run on a host (or container with a higher cgroup limit) with more RAM.
4. As a last resort, set `GITNEXUS_MEMORY=off` to suppress the guard (proceed at your own risk — expect long GC stalls or a hard OOM).

Example fix

// before — CI pins the heap below machine RAM
NODE_OPTIONS='--max-old-space-size=2048' gitnexus analyze big-monorepo
# → Analyze stopped before running out of memory: 1888MB of the 2048MB Node heap...

// after — drop the pin; let gitnexus size itself
gitnexus analyze big-monorepo

// Estimate heap need and warn before running analyze
import { projectParseHeapNeedBytes } from 'gitnexus/dist/core/ingestion/pipeline-phases/parse-impl.js';
import v8 from 'node:v8';

const heapLimit = v8.getHeapStatistics().heap_size_limit;
const projected = projectParseHeapNeedBytes(parseableFileCount);
if (projected > heapLimit * 0.92) {
  throw new Error(
    `Projected parse heap (${Math.round(projected/1024/1024)}MB) exceeds 92% of the ` +
    `heap limit (${Math.round(heapLimit/1024/1024)}MB). Raise --max-old-space-size ` +
    `or add a .gitnexusignore.`,
  );
}

try {
  await analyze(repo, options);
} catch (err) {
  if (/Analyze stopped before running out of memory/.test(err.message)) {
    // The message names the remedy (drop the pin, or .gitnexusignore, or more RAM).
    // Act on it, then re-run. Do NOT retry unchanged.
    console.error(err.message);
    process.exit(3);
  }
  throw err;
}

When it happens

Trigger: Analyzing a repo whose combined AST/node set does not fit in the process heap; the guard fires at a chunk boundary (chunkIdx>0 region) when `shouldAbortForHeapPressure` returns true. Most common when `--max-old-space-size` (or a NODE_OPTIONS pin) caps the heap below what the machine actually has, or when the repo contains very large generated/vendored trees.

Common situations: A CI runner with a 2GB NODE_OPTIONS cap on a monorepo with a huge `node_modules`/`vendor` dir. A container whose cgroup memory limit is well below host RAM. A repo that bundles generated code (protobuf stubs, auto-generated grammar files) that bloats the node count.

Related errors

• Worker-pool parsing cannot be disabled (${reason}). GitNexus
• Worker pool failed to start: ${cause}${failureDetail}\n\nThe
• Worker script not found: ${workerPath}
• Worker pool circuit breaker tripped${reason}. Subsequent dis
• Worker pool has no active workers${detail}

AI-assisted analysis of abhigyanpatwari/GitNexus@d540b00184 (2026-08-12). Data as JSON: /api/errors/53a0ed6edfc61d1c. Report an issue: GitHub.