abhigyanpatwari/GitNexus · critical · Error

GraphEmitSink: ${errors.length} streamed CSV writer(s) hit a

Error message

GraphEmitSink: ${errors.length} streamed CSV writer(s) hit an IO error (disk-full / out-of-fds) during the emit — the persisted graph would be truncated, so the run is failed rather than COPYing a partial CSV: ${first}

What it means

Thrown by `GraphEmitSink.finalize()` when one or more streamed CSV writers recorded an IO fault during the graph emit. Each per-relationship-pair writer sets a `poison` on any write/flush/open failure; `finalize()` closes all writers, collects the poisoned ones (plus any `openFailure`), and if any exist it fails the run rather than handing a truncated CSV to the bulk COPY. The design is explicit: a partial persisted graph is worse than a clean failure.

Source

Thrown at gitnexus/src/core/lbug/graph-emit-sink.ts:501

  finalize(): GraphEmitManifest {
    if (this.finalized) throw new Error('GraphEmitSink.finalize() called twice');
    this.finalized = true;

    const errors: unknown[] = [];
    if (this.openFailure !== undefined) errors.push(this.openFailure);

    const relsByPair = new Map<string, { csvPath: string; rows: number }>();
    let totalRows = 0;
    for (const [pairKey, writer] of this.relWriters) {
      writer.close();
      if (writer.poison !== undefined) errors.push(writer.poison);
      relsByPair.set(pairKey, { csvPath: writer.csvPath, rows: writer.rows });
      totalRows += writer.rows;
    }

    if (errors.length > 0) {
      const first = errors[0];
      throw new Error(
        `GraphEmitSink: ${errors.length} streamed CSV writer(s) hit an IO error ` +
          `(disk-full / out-of-fds) during the emit — the persisted graph would be ` +
          `truncated, so the run is failed rather than COPYing a partial CSV: ${
            first instanceof Error ? first.message : String(first)
          }`,
      );
    }

    return { relsByPair, totalRows, structuralRows: this.structuralRows };
  }

  /** Best-effort fd release for the error path — when the pipeline throws
   *  before {@link finalize} runs, the caller's `finally` calls this so the
   *  per-pair fds never leak. Idempotent with finalize via `finalized`. */
  close(): void {
    if (this.finalized) return;
    this.finalized = true;
    for (const writer of this.relWriters.values()) {

View on GitHub (pinned to d540b00184)

Solutions

  1. Free disk space on the volume holding the GitNexus storage directory (`.gitnexus/`).
  2. Raise the file-descriptor limit (`ulimit -n 65536`) before running analyze, especially in containers.
  3. Move the storage directory to a local non-overlay filesystem if you are on a network mount.
  4. Re-run `gitnexus analyze --force` after freeing resources — the emit is not transactional and a partial CSV may have been written.

Example fix

// before — CI defaults to a low fd limit
ulimit -n 1024
# → GraphEmitSink: 3 streamed CSV writer(s) hit an IO error (out-of-fds)...

// after
ulimit -n 65536
gitnexus analyze --force repo
Defensive patterns

Strategy: validation

Validate before calling

import { checkDiskSpace } from './disk-utils.js'; // your helper

// Before analyze, ensure the storage volume has headroom and fds are plentiful
const free = await checkDiskSpace(storageDir);
const minBytes = estimatedGraphBytes * 2; // CSVs + DB
if (free < minBytes) {
  throw new Error(`Insufficient disk space: ${free} bytes free, need ~${minBytes}.`);
}
// Raise fd limit if the repo will open many relationship-pair writers
if (Number(process.env.UV_THREADPOOL_SIZE ?? 4) < 16 && manyRelPairs) {
  throw new Error('Raise ulimit -n before emitting a wide relationship set.');
}

Try / catch

try {
  sink.finalize();
} catch (err) {
  if (/GraphEmitSink.*IO error/i.test(err.message)) {
    // disk-full / out-of-fds: free space + raise ulimit, then --force rebuild.
    console.error(err.message, '— free disk, raise `ulimit -n`, re-run with --force');
    process.exit(5);
  }
  throw err;
}

Prevention

When it happens

Trigger: A disk-full condition, out-of-file-descriptors (EMFILE/ENFILE), a write/flush error, or a writer-open failure during the relationship CSV emit — any of these poison a writer, and `finalize()` surfaces the aggregate count with the first error's message.

Common situations: Analyzing a very large repo that exhausts disk space in the staging dir; a CI/container with a low `ulimit -n` that runs out of fds because there is one writer per relationship pair; a network/overlay filesystem that errors under heavy write load.

Related errors

AI-assisted analysis of abhigyanpatwari/GitNexus@d540b00184 (2026-08-12). Data as JSON: /api/errors/5562014cbb3ad25c. Report an issue: GitHub.