{"record":{"id":"5562014cbb3ad25c","repo":"abhigyanpatwari/GitNexus","slug":"graphemitsink-errors-length-streamed-csv-write","errorCode":null,"errorMessage":"GraphEmitSink: ${errors.length} streamed CSV writer(s) hit an IO error (disk-full / out-of-fds) during the emit — the persisted graph would be truncated, so the run is failed rather than COPYing a partial CSV: ${first}","messagePattern":"GraphEmitSink: (.+?) streamed CSV writer\\(s\\) hit an IO error \\(disk-full / out-of-fds\\) during the emit — the persisted graph would be truncated, so the run is failed rather than COPYing a partial CSV: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"critical","filePath":"gitnexus/src/core/lbug/graph-emit-sink.ts","lineNumber":501,"sourceCode":"  finalize(): GraphEmitManifest {\n    if (this.finalized) throw new Error('GraphEmitSink.finalize() called twice');\n    this.finalized = true;\n\n    const errors: unknown[] = [];\n    if (this.openFailure !== undefined) errors.push(this.openFailure);\n\n    const relsByPair = new Map<string, { csvPath: string; rows: number }>();\n    let totalRows = 0;\n    for (const [pairKey, writer] of this.relWriters) {\n      writer.close();\n      if (writer.poison !== undefined) errors.push(writer.poison);\n      relsByPair.set(pairKey, { csvPath: writer.csvPath, rows: writer.rows });\n      totalRows += writer.rows;\n    }\n\n    if (errors.length > 0) {\n      const first = errors[0];\n      throw new Error(\n        `GraphEmitSink: ${errors.length} streamed CSV writer(s) hit an IO error ` +\n          `(disk-full / out-of-fds) during the emit — the persisted graph would be ` +\n          `truncated, so the run is failed rather than COPYing a partial CSV: ${\n            first instanceof Error ? first.message : String(first)\n          }`,\n      );\n    }\n\n    return { relsByPair, totalRows, structuralRows: this.structuralRows };\n  }\n\n  /** Best-effort fd release for the error path — when the pipeline throws\n   *  before {@link finalize} runs, the caller's `finally` calls this so the\n   *  per-pair fds never leak. Idempotent with finalize via `finalized`. */\n  close(): void {\n    if (this.finalized) return;\n    this.finalized = true;\n    for (const writer of this.relWriters.values()) {","sourceCodeStart":483,"sourceCodeEnd":519,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/d540b00184d71a896261ee02670da9a92d59d8f7/gitnexus/src/core/lbug/graph-emit-sink.ts#L483-L519","documentation":"Thrown by `GraphEmitSink.finalize()` when one or more streamed CSV writers recorded an IO fault during the graph emit. Each per-relationship-pair writer sets a `poison` on any write/flush/open failure; `finalize()` closes all writers, collects the poisoned ones (plus any `openFailure`), and if any exist it fails the run rather than handing a truncated CSV to the bulk COPY. The design is explicit: a partial persisted graph is worse than a clean failure.","triggerScenarios":"A disk-full condition, out-of-file-descriptors (EMFILE/ENFILE), a write/flush error, or a writer-open failure during the relationship CSV emit — any of these poison a writer, and `finalize()` surfaces the aggregate count with the first error's message.","commonSituations":"Analyzing a very large repo that exhausts disk space in the staging dir; a CI/container with a low `ulimit -n` that runs out of fds because there is one writer per relationship pair; a network/overlay filesystem that errors under heavy write load.","solutions":["Free disk space on the volume holding the GitNexus storage directory (`.gitnexus/`).","Raise the file-descriptor limit (`ulimit -n 65536`) before running analyze, especially in containers.","Move the storage directory to a local non-overlay filesystem if you are on a network mount.","Re-run `gitnexus analyze --force` after freeing resources — the emit is not transactional and a partial CSV may have been written."],"exampleFix":"// before — CI defaults to a low fd limit\nulimit -n 1024\n# → GraphEmitSink: 3 streamed CSV writer(s) hit an IO error (out-of-fds)...\n\n// after\nulimit -n 65536\ngitnexus analyze --force repo","handlingStrategy":"validation","validationCode":"import { checkDiskSpace } from './disk-utils.js'; // your helper\n\n// Before analyze, ensure the storage volume has headroom and fds are plentiful\nconst free = await checkDiskSpace(storageDir);\nconst minBytes = estimatedGraphBytes * 2; // CSVs + DB\nif (free < minBytes) {\n  throw new Error(`Insufficient disk space: ${free} bytes free, need ~${minBytes}.`);\n}\n// Raise fd limit if the repo will open many relationship-pair writers\nif (Number(process.env.UV_THREADPOOL_SIZE ?? 4) < 16 && manyRelPairs) {\n  throw new Error('Raise ulimit -n before emitting a wide relationship set.');\n}","typeGuard":null,"tryCatchPattern":"try {\n  sink.finalize();\n} catch (err) {\n  if (/GraphEmitSink.*IO error/i.test(err.message)) {\n    // disk-full / out-of-fds: free space + raise ulimit, then --force rebuild.\n    console.error(err.message, '— free disk, raise `ulimit -n`, re-run with --force');\n    process.exit(5);\n  }\n  throw err;\n}","preventionTips":["Raise `ulimit -n` (e.g. 65536) in CI/containers before analyzing large repos.","Ensure the storage volume has at least 2x the estimated graph size free.","Keep the storage directory on a local filesystem, not a network/overlay mount."],"tags":["io","disk-full","file-descriptors","csv","graph-emit","emit"],"backgroundTag":null,"analyzedSha":"d540b00184d71a896261ee02670da9a92d59d8f7","analyzedAt":"2026-08-12T19:50:25.132Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}