abhigyanpatwari/GitNexus · error · Error

COPY failed for ${table}: ${retryMsg}${remedy}

Error message

COPY failed for ${table}: ${retryMsg}${remedy}

What it means

Thrown by `copyNodeCSVs` when the bulk `COPY` of a node CSV into LadybugDB exhausts its retries (`copyCsvWithRetry`). The error message includes the table name, the truncated retry message, and — when `bufferPoolExhaustionRemedy` matches a pool-exhaustion signature (#2631) — an actionable remedy naming the `GITNEXUS_LBUG_BUFFER_POOL_SIZE` knob and, on non-4K-page hosts, the page-granule amplification factor. The load is NOT TRANSACTIONAL: a failed COPY leaves a partially-loaded DB and recovery is a `--force` rebuild, not a partial retry.

Source

Thrown at gitnexus/src/core/lbug/lbug-adapter.ts:1066

  log: (message: string) => void,
  totalSteps: number,
): Promise<void> => {
  let stepsDone = 0;
  for (const [table, { csvPath, rows }] of nodeFileEntries) {
    stepsDone++;
    log(`Loading nodes ${stepsDone}/${totalSteps}: ${table} (${rows.toLocaleString()} rows)`);

    if (!(await stagingCsvExists(csvPath))) throw missingStagingCsvError(table, csvPath, rows);

    const copyQuery = getCopyQuery(table, normalizeCopyPath(csvPath));
    await copyCsvWithRetry(targetConn, copyQuery, (retryErr) => {
      const retryMsg = retryErr instanceof Error ? retryErr.message : String(retryErr);
      // Pool exhaustion gets a remedy (#2631): the raw binder text gives the
      // operator nothing to act on, and on non-4K-page hosts (Ascend aarch64,
      // Apple Silicon) the pool bills up to pageSize/4KiB x faster than the
      // sizing was calibrated for — name the knob and the mechanism.
      const remedy = bufferPoolExhaustionRemedy(retryMsg);
      throw new Error(
        `COPY failed for ${table}: ${retryMsg.slice(0, 200)}${remedy ? ` ${remedy}` : ''}`,
      );
    });
  }
};

/**
 * Persist a KnowledgeGraph: stream CSVs, then bulk-COPY nodes (overlapped with
 * relationship emit — see the body) and relationships.
 *
 * NOT TRANSACTIONAL (#2226). Each `COPY` commits independently and there is no
 * surrounding transaction, so a failure partway through — a node `COPY` that
 * throws at the FK barrier, a relationship `COPY` failure, or a `pdgEmitManifest`
 * collision raised after node rows have already committed in the overlap path —
 * leaves a partially-loaded DB. The caller surfaces the error; recovery is a
 * `--force` re-analyze (a full rebuild), not a partial retry. Callers must not
 * assume the DB is either fully loaded or untouched after a rejection.
 */

View on GitHub (pinned to d540b00184)

Solutions

1. If the remedy text mentions the buffer pool: raise it via `GITNEXUS_LBUG_BUFFER_POOL_SIZE=<bytes>` (e.g. 4 GiB = 4294967296); set to 0 for LadybugDB's native 80%-of-RAM default.
2. Re-run `gitnexus analyze --force` (the DB is partially loaded after a COPY failure — do not assume it is usable).
3. Free RAM / reduce other memory pressure so the buffer pool can grow.
4. Verify the staging CSV exists and is complete (an earlier emit failure could truncate it).

Example fix

# before — default pool exhausted on a 64KiB-page aarch64 host
gitnexus analyze big-repo
# → COPY failed for Symbol: ... buffer pool exhausted...

# after — raise the buffer pool
GITNEXUS_LBUG_BUFFER_POOL_SIZE=4294967296 gitnexus analyze --force big-repo

// Before analyze, check that the buffer pool is sized for a non-4K-page host
import { getOsPageSize } from './os-utils.js'; // your helper

const pageSize = getOsPageSize();
if (pageSize > 4096 && !process.env.GITNEXUS_LBUG_BUFFER_POOL_SIZE) {
  const ratio = pageSize / 4096;
  console.warn(
    `OS page size ${pageSize/1024}KiB amplifies pool use ~${ratio}x; ` +
    `set GITNEXUS_LBUG_BUFFER_POOL_SIZE for large repos.`,
  );
}

function isCopyBufferPoolError(err): boolean {
  return /COPY failed for .* buffer pool/i.test(
    err instanceof Error ? err.message : String(err),
  );
}

// On buffer-pool exhaustion, raise the pool and rebuild (the DB is partially loaded).
try {
  await loadGraphToLbug(graph, repoPath, storagePath);
} catch (err) {
  if (/COPY failed for .* buffer pool/i.test(err.message)) {
    process.env.GITNEXUS_LBUG_BUFFER_POOL_SIZE = String(4 * 1024 * 1024 * 1024);
    // NOT TRANSACTIONAL — must --force rebuild, not resume.
    await runForceRebuild(repoPath);
    return;
  }
  throw err;
}

When it happens

Trigger: A `COPY <table> FROM '<csv>'` query fails after retry during `loadGraphToLbug`. Most commonly LadybugDB buffer-pool exhaustion (the pool bills memory faster on non-4K-page hosts like aarch64/Apple Silicon), but also covers any COPY-time binder/IO error.

Common situations: Analyzing a large repo on an aarch64/Apple-Silicon host where 64KiB pages amplify pool usage up to 16x vs the 4KiB calibration; a host with the default buffer pool too small for the node count; a staging CSV that is incomplete or unreadable.

Related errors

• LadybugDB checkpoint sidecar is missing for ${dbPath}. Rebui
• LadybugDB WAL corruption detected at ${dbPath}. WAL corrupti
• [embed] Failed to delete stale embedding rows — aborting to
• Bridge query prepare failed: ${errMsg}
• Bridge query returned an empty QueryResult array

AI-assisted analysis of abhigyanpatwari/GitNexus@d540b00184 (2026-08-12). Data as JSON: /api/errors/2b0121db80ddd147. Report an issue: GitHub.