{"record":{"id":"3194a8c70547a272","repo":"abhigyanpatwari/GitNexus","slug":"pdgemitsink-errors-length-streamed-csv-writer","errorCode":null,"errorMessage":"PdgEmitSink: ${errors.length} streamed CSV writer(s) hit an IO error (disk-full / out-of-fds) during the emit — the persisted graph would be truncated, so the run is failed rather than COPYing a partial CSV: ${first instanceof Error ? first.message : String(first)}","messagePattern":"PdgEmitSink: (.+?) streamed CSV writer\\(s\\) hit an IO error \\(disk-full / out-of-fds\\) during the emit — the persisted graph would be truncated, so the run is failed rather than COPYing a partial CSV: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"gitnexus/src/core/lbug/pdg-emit-sink.ts","lineNumber":239,"sourceCode":"    if (this.bbWriter !== undefined) {\n      this.bbWriter.close();\n      if (this.bbWriter.poison !== undefined) errors.push(this.bbWriter.poison);\n      nodeFiles.set('BasicBlock' as NodeTableName, {\n        csvPath: this.bbWriter.csvPath,\n        rows: this.bbWriter.rows,\n      });\n    }\n\n    const relsByPair = new Map<string, { csvPath: string; rows: number }>();\n    for (const [pairKey, writer] of this.relWriters) {\n      writer.close();\n      if (writer.poison !== undefined) errors.push(writer.poison);\n      relsByPair.set(pairKey, { csvPath: writer.csvPath, rows: writer.rows });\n    }\n\n    if (errors.length > 0) {\n      const first = errors[0];\n      throw new Error(\n        `PdgEmitSink: ${errors.length} streamed CSV writer(s) hit an IO error ` +\n          `(disk-full / out-of-fds) during the emit — the persisted graph would ` +\n          `be truncated, so the run is failed rather than COPYing a partial CSV: ${\n            first instanceof Error ? first.message : String(first)\n          }`,\n      );\n    }\n\n    return { nodeFiles, relsByPair };\n  }\n\n  /**\n   * Best-effort fd release for the error path — when a language pass throws\n   * before {@link finalize} runs, the caller's `finally` calls this so the\n   * BasicBlock + per-pair fds never leak. Idempotent with finalize via the\n   * `finalized` flag; close errors are swallowed because the run is already\n   * failing.\n   */","sourceCodeStart":221,"sourceCodeEnd":257,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/d540b00184d71a896261ee02670da9a92d59d8f7/gitnexus/src/core/lbug/pdg-emit-sink.ts#L221-L257","documentation":"Thrown by PdgEmitSink.finalize() when one or more streamed CSV writers recorded a poison error (IO failure) during the PDG (Program Dependence Graph) emit phase. The sink streams node and relationship data to CSV files that are later COPY'd into LadybugDB; if any writer hits disk-full or out-of-file-descriptors mid-stream, the persisted graph would be truncated. Rather than allowing a partial CSV to be loaded (producing a silently incomplete PDG), the entire run is failed. Each CsvWriter stores its error in a poison field checked at close time.","triggerScenarios":"Calling PdgEmitSink.finalize() after writing PDG nodes/edges when the filesystem runs out of space (disk-full) or the process exceeds its file descriptor limit (ulimit -n). The error aggregates all poisoned writers and reports the first one's message. Each relWriters entry (keyed by source-label:target-label pair) and nodeWriter can independently fail.","commonSituations":"Analyzing a very large repository that generates millions of PDG edges, filling the disk during CSV streaming; running in a CI container with a low ulimit -n (e.g. 1024) where many CSV files exhaust file descriptors; a mounted network filesystem with quota limits; running concurrent analyzes that share a disk partition.","solutions":["Free disk space on the volume holding the .gitnexus/ storage directory — PDG CSVs can be several GB for large repos","Raise the file descriptor limit: `ulimit -n 65536` before running `gitnexus analyze`, or set it system-wide in /etc/security/limits.conf","Re-run `gitnexus analyze` — the failed PDG emit leaves the dirty flag set, triggering a clean rebuild","If the repo is extremely large, consider analyzing a subset or increasing available disk to at least 3x the repository size","Check `df -h` and `ulimit -n` before starting large analyzes to catch these conditions early"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Check disk space and file descriptor limit before PDG emit\nimport { statfs } from 'fs/promises';\nasync function preflightPdgEmit(outputDir: string): Promise<void> {\n  const stats = await statfs(outputDir);\n  const freeGB = (stats.bavail * stats.bsize) / (1024 ** 3);\n  if (freeGB < 1) {\n    throw new Error(`Insufficient disk space: ${freeGB.toFixed(2)} GB free — need >= 1 GB for PDG emit`);\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  const result = await pdgEmitSink.finalize();\n} catch (e) {\n  if (e instanceof Error && e.message.startsWith('PdgEmitSink:')) {\n    // IO error — check disk space and ulimit, then re-run analyze\n    logger.error('PDG emit IO failure — check df -h and ulimit -n', e);\n  }\n  throw e;\n}","preventionTips":["Check `df -h` and ensure at least 3x the repo size in free disk space before analyzing large repos","Set `ulimit -n 65536` (or higher) before running analyze on large repos","In Docker, set `--ulimit nofile=65536:65536` on the container","Monitor disk usage during long-running analyzes — PDG CSVs can be several GB"],"tags":["pdg","csv-writer","disk-full","file-descriptors","emit","io-error"],"backgroundTag":null,"analyzedSha":"d540b00184d71a896261ee02670da9a92d59d8f7","analyzedAt":"2026-08-12T19:50:25.132Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}