{"record":{"id":"2b0121db80ddd147","repo":"abhigyanpatwari/GitNexus","slug":"copy-failed-for-table-retrymsg-remedy","errorCode":null,"errorMessage":"COPY failed for ${table}: ${retryMsg}${remedy}","messagePattern":"COPY failed for (.+?): (.+?)(.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"gitnexus/src/core/lbug/lbug-adapter.ts","lineNumber":1066,"sourceCode":"  log: (message: string) => void,\n  totalSteps: number,\n): Promise<void> => {\n  let stepsDone = 0;\n  for (const [table, { csvPath, rows }] of nodeFileEntries) {\n    stepsDone++;\n    log(`Loading nodes ${stepsDone}/${totalSteps}: ${table} (${rows.toLocaleString()} rows)`);\n\n    if (!(await stagingCsvExists(csvPath))) throw missingStagingCsvError(table, csvPath, rows);\n\n    const copyQuery = getCopyQuery(table, normalizeCopyPath(csvPath));\n    await copyCsvWithRetry(targetConn, copyQuery, (retryErr) => {\n      const retryMsg = retryErr instanceof Error ? retryErr.message : String(retryErr);\n      // Pool exhaustion gets a remedy (#2631): the raw binder text gives the\n      // operator nothing to act on, and on non-4K-page hosts (Ascend aarch64,\n      // Apple Silicon) the pool bills up to pageSize/4KiB x faster than the\n      // sizing was calibrated for — name the knob and the mechanism.\n      const remedy = bufferPoolExhaustionRemedy(retryMsg);\n      throw new Error(\n        `COPY failed for ${table}: ${retryMsg.slice(0, 200)}${remedy ? ` ${remedy}` : ''}`,\n      );\n    });\n  }\n};\n\n/**\n * Persist a KnowledgeGraph: stream CSVs, then bulk-COPY nodes (overlapped with\n * relationship emit — see the body) and relationships.\n *\n * NOT TRANSACTIONAL (#2226). Each `COPY` commits independently and there is no\n * surrounding transaction, so a failure partway through — a node `COPY` that\n * throws at the FK barrier, a relationship `COPY` failure, or a `pdgEmitManifest`\n * collision raised after node rows have already committed in the overlap path —\n * leaves a partially-loaded DB. The caller surfaces the error; recovery is a\n * `--force` re-analyze (a full rebuild), not a partial retry. Callers must not\n * assume the DB is either fully loaded or untouched after a rejection.\n */","sourceCodeStart":1048,"sourceCodeEnd":1084,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/d540b00184d71a896261ee02670da9a92d59d8f7/gitnexus/src/core/lbug/lbug-adapter.ts#L1048-L1084","documentation":"Thrown by `copyNodeCSVs` when the bulk `COPY` of a node CSV into LadybugDB exhausts its retries (`copyCsvWithRetry`). The error message includes the table name, the truncated retry message, and — when `bufferPoolExhaustionRemedy` matches a pool-exhaustion signature (#2631) — an actionable remedy naming the `GITNEXUS_LBUG_BUFFER_POOL_SIZE` knob and, on non-4K-page hosts, the page-granule amplification factor. The load is NOT TRANSACTIONAL: a failed COPY leaves a partially-loaded DB and recovery is a `--force` rebuild, not a partial retry.","triggerScenarios":"A `COPY <table> FROM '<csv>'` query fails after retry during `loadGraphToLbug`. Most commonly LadybugDB buffer-pool exhaustion (the pool bills memory faster on non-4K-page hosts like aarch64/Apple Silicon), but also covers any COPY-time binder/IO error.","commonSituations":"Analyzing a large repo on an aarch64/Apple-Silicon host where 64KiB pages amplify pool usage up to 16x vs the 4KiB calibration; a host with the default buffer pool too small for the node count; a staging CSV that is incomplete or unreadable.","solutions":["If the remedy text mentions the buffer pool: raise it via `GITNEXUS_LBUG_BUFFER_POOL_SIZE=<bytes>` (e.g. 4 GiB = 4294967296); set to 0 for LadybugDB's native 80%-of-RAM default.","Re-run `gitnexus analyze --force` (the DB is partially loaded after a COPY failure — do not assume it is usable).","Free RAM / reduce other memory pressure so the buffer pool can grow.","Verify the staging CSV exists and is complete (an earlier emit failure could truncate it)."],"exampleFix":"# before — default pool exhausted on a 64KiB-page aarch64 host\ngitnexus analyze big-repo\n# → COPY failed for Symbol: ... buffer pool exhausted...\n\n# after — raise the buffer pool\nGITNEXUS_LBUG_BUFFER_POOL_SIZE=4294967296 gitnexus analyze --force big-repo","handlingStrategy":"retry","validationCode":"// Before analyze, check that the buffer pool is sized for a non-4K-page host\nimport { getOsPageSize } from './os-utils.js'; // your helper\n\nconst pageSize = getOsPageSize();\nif (pageSize > 4096 && !process.env.GITNEXUS_LBUG_BUFFER_POOL_SIZE) {\n  const ratio = pageSize / 4096;\n  console.warn(\n    `OS page size ${pageSize/1024}KiB amplifies pool use ~${ratio}x; ` +\n    `set GITNEXUS_LBUG_BUFFER_POOL_SIZE for large repos.`,\n  );\n}","typeGuard":"function isCopyBufferPoolError(err): boolean {\n  return /COPY failed for .* buffer pool/i.test(\n    err instanceof Error ? err.message : String(err),\n  );\n}","tryCatchPattern":"// On buffer-pool exhaustion, raise the pool and rebuild (the DB is partially loaded).\ntry {\n  await loadGraphToLbug(graph, repoPath, storagePath);\n} catch (err) {\n  if (/COPY failed for .* buffer pool/i.test(err.message)) {\n    process.env.GITNEXUS_LBUG_BUFFER_POOL_SIZE = String(4 * 1024 * 1024 * 1024);\n    // NOT TRANSACTIONAL — must --force rebuild, not resume.\n    await runForceRebuild(repoPath);\n    return;\n  }\n  throw err;\n}","preventionTips":["On aarch64/Apple-Silicon (64KiB pages), pre-set GITNEXUS_LBUG_BUFFER_POOL_SIZE for large repos.","Always re-run with --force after a COPY failure — the DB is partially loaded.","Ensure the staging CSV emit completed fully before the COPY (watch for earlier emit errors)."],"tags":["ladybugdb","copy","buffer-pool","bulk-load","memory","rebuild"],"backgroundTag":null,"analyzedSha":"d540b00184d71a896261ee02670da9a92d59d8f7","analyzedAt":"2026-08-12T19:50:25.132Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}