{"record":{"id":"e8d7ffb18802cec6","repo":"abhigyanpatwari/GitNexus","slug":"cobol-processor-skipping-oversized-file-file","errorCode":null,"errorMessage":"[cobol-processor] Skipping oversized file (${(file.content.length / 1024 / 1024).toFixed(1)}MB > ${(MAX_COBOL_FILE_SIZE / 1024 / 1024).toFixed(0)}MB): ${file.path}","messagePattern":"\\[cobol-processor\\] Skipping oversized file \\((.+?)MB > (.+?)MB\\): (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"gitnexus/src/core/ingestion/cobol-processor.ts","lineNumber":187,"sourceCode":"    const cached = preprocessedCopyCache.get(copyPath);\n    if (cached !== undefined) return cached;\n    const content = copybookByPath.get(copyPath);\n    if (!content) return null; // preserves original falsy→null (missing/empty)\n    const preprocessed = preprocessCobolSource(content);\n    preprocessedCopyCache.set(copyPath, preprocessed);\n    return preprocessed;\n  };\n\n  // Track module names for cross-program CALL resolution\n  const moduleNodeIds = new Map<string, string>(); // uppercase program name -> node id\n\n  // ── 3. Process each COBOL program ──────────────────────────────────\n  const raw = parseInt(process.env.GITNEXUS_MAX_COBOL_FILE_SIZE_BYTES ?? '', 10);\n  const MAX_COBOL_FILE_SIZE = Number.isFinite(raw) && raw > 0 ? raw : 5 * 1024 * 1024;\n  for (const file of programs) {\n    // File-size guard: skip excessively large files to prevent OOM\n    if (file.content.length > MAX_COBOL_FILE_SIZE) {\n      logger.warn(\n        `[cobol-processor] Skipping oversized file (${(file.content.length / 1024 / 1024).toFixed(1)}MB > ${(MAX_COBOL_FILE_SIZE / 1024 / 1024).toFixed(0)}MB): ${file.path}`,\n      );\n      continue;\n    }\n    const fileNodeId = generateId('File', file.path);\n    // Skip if file node doesn't exist (structure-processor creates it)\n    if (!graph.getNode(fileNodeId)) continue;\n\n    // Preprocess: clean patch markers\n    const cleaned = preprocessCobolSource(file.content);\n\n    // Expand COPY statements\n    const { expandedContent, copyResolutions } = expandCopies(\n      cleaned,\n      file.path,\n      resolveCopy,\n      readCopy,\n    );","sourceCodeStart":169,"sourceCodeEnd":205,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/aac7515d2a8c50a1f8f923c6fb77218b333560d6/gitnexus/src/core/ingestion/cobol-processor.ts#L169-L205","documentation":"The COBOL processor guards against OOM by skipping any program file whose content length exceeds MAX_COBOL_FILE_SIZE — 5 MiB by default, overridable via GITNEXUS_MAX_COBOL_FILE_SIZE_BYTES (raw bytes; invalid or non-positive values fall back to 5 MiB). The file is skipped entirely: no node processing, no preprocessing, no COPY expansion for it.","triggerScenarios":"A COBOL source larger than the cap — often a single member concatenating many programs or a generated file; or the env var set lower than a legitimate file's size.","commonSituations":"Mainframe exports with many programs per member; generated COBOL; environments that set GITNEXUS_MAX_COBOL_FILE_SIZE_BYTES conservatively and then index large legacy members.","solutions":["Raise the cap if memory allows: export GITNEXUS_MAX_COBOL_FILE_SIZE_BYTES=20971520 (20 MiB)","Split the oversized member into one program per file","Exclude the file from indexing if it is not needed"],"exampleFix":"# before\n# default cap 5 MiB; big.cbl (12 MiB) is skipped\n\n# after\nexport GITNEXUS_MAX_COBOL_FILE_SIZE_BYTES=20971520\nnode .gitnexus/run.cjs analyze --index-only","handlingStrategy":"validation","validationCode":"import { statSync } from 'node:fs';\n\nconst cap = (() => {\n  const raw = parseInt(process.env.GITNEXUS_MAX_COBOL_FILE_SIZE_BYTES ?? '', 10);\n  return Number.isFinite(raw) && raw > 0 ? raw : 5 * 1024 * 1024;\n})();\nconst oversized = cobolFiles.filter((f) => statSync(f).size > cap);\nif (oversized.length > 0) {\n  console.warn(`will be skipped (>${cap} bytes): ${oversized.join(', ')}`);\n  // decide: raise the env cap, split the member, or exclude these files\n}","typeGuard":"function isWithinCobolCap(sizeBytes: number, cap = 5 * 1024 * 1024): boolean {\n  return sizeBytes <= cap;\n}","tryCatchPattern":null,"preventionTips":["Split multi-program COBOL members into one program per file before indexing","Size-check large members up front and set GITNEXUS_MAX_COBOL_FILE_SIZE_BYTES to a value your memory budget allows","Watch the analyzer's memory headroom when raising the cap — the guard exists to prevent OOM"],"tags":["cobol","file-size","oom-guard","env-var"],"backgroundTag":"file-size-limit-exceeded","analyzedSha":"aac7515d2a8c50a1f8f923c6fb77218b333560d6","analyzedAt":"2026-08-20T23:29:22.980Z","contentChangedAt":"2026-08-20T23:29:22.980Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}