{"record":{"id":"40c3237d7a410bee","repo":"abhigyanpatwari/GitNexus","slug":"incremental-importer-bfs-dropped-chunk-chunkind","errorCode":null,"errorMessage":"Incremental importer BFS: dropped chunk ${chunkIndex} (${batch.length} target path(s)) — importer expansion degrades for this run; affected importers may keep stale edges until the next full rebuild.","messagePattern":"Incremental importer BFS: dropped chunk (.+?) \\((.+?) target path\\(s\\)\\) — importer expansion degrades for this run; affected importers may keep stale edges until the next full rebuild\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"gitnexus/src/core/lbug/lbug-adapter.ts","lineNumber":2719,"sourceCode":"      RETURN DISTINCT a.filePath AS importer\n    `;\n    await withConnLock(async () => {\n      let queryResult: lbug.QueryResult | lbug.QueryResult[] | undefined;\n      try {\n        queryResult = await c.query(cypher);\n        const result = Array.isArray(queryResult) ? queryResult[0] : queryResult;\n        const rows = await result.getAll();\n        for (const row of rows) {\n          const v = (row as { importer?: unknown }).importer;\n          if (typeof v === 'string' && v.length > 0) importers.add(v);\n        }\n      } catch (err) {\n        // Degrade-don't-fail, mirroring queryImporters — but LOUDLY\n        // (tri-review 4669518496 P2-5): a dropped chunk means every importer\n        // it would have surfaced keeps possibly-stale edges this run, and the\n        // old bare `catch {}` left no trace of that anywhere. pino idiom:\n        // `err` key — `error` serializes to `{}`.\n        logger.warn(\n          { err },\n          `Incremental importer BFS: dropped chunk ${chunkIndex} (${batch.length} target path(s)) — ` +\n            'importer expansion degrades for this run; affected importers may keep stale edges until the next full rebuild.',\n        );\n        options.onChunkFailure?.(chunkIndex, batch.length, err);\n      } finally {\n        if (queryResult) await closeQueryResults(queryResult);\n      }\n    });\n  }\n  // Cypher without ORDER BY is unordered — sort so downstream chunking and\n  // logs are stable run-to-run (matches diffFileHashes' sorted outputs).\n  return [...importers].sort();\n};\n\n/**\n * Drop every Community and Process node (and their MEMBER_OF /\n * STEP_IN_PROCESS edges via DETACH DELETE). Used at the start of an","sourceCodeStart":2701,"sourceCodeEnd":2737,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/aac7515d2a8c50a1f8f923c6fb77218b333560d6/gitnexus/src/core/lbug/lbug-adapter.ts#L2701-L2737","documentation":"The incremental importer BFS expands 'who imports the changed files' in batched queries; one chunk's query failed, so every importer those target paths would have surfaced is dropped from the expansion set. Consequence stated in the message: affected importers can keep stale edges until the next full rebuild. Loud by design (the old bare catch{} left no trace), and it notifies options.onChunkFailure(chunkIndex, batch.length, err).","triggerScenarios":"collectImportersForFiles-style BFS runs during incremental analyze; a per-chunk Cypher query throws (lock timeout, transient I/O, connection blip). The catch degrades-don't-fails for that chunk, logs with pino's `err` key (not `error`, which serializes to {}), and moves to the next chunk.","commonSituations":"Incremental runs racing a long serve query holding locks; flaky network filesystems under the DB; memory pressure causing native query faults; very large batches amplifying query timeouts.","solutions":["Re-run the incremental analyze — the next run re-queries all chunks and usually completes the expansion.","Wire/surface options.onChunkFailure in your orchestration to count degraded runs and trigger repair.","Stop concurrent serve/MCP traffic against the DB during incremental updates to remove lock contention.","If stale edges are suspected, schedule a full rebuild (`analyze --force`) to reconcile."],"exampleFix":"// before: chunk failure invisible, stale edges linger unexplained\nawait deleteNodesForFiles(...);\n\n// after: observe degraded chunks and repair\nawait deleteNodesForFiles(targets, {\n  onChunkFailure: (chunkIndex, count, err) => {\n    metrics.incr('importer_bfs_dropped');\n    log.warn({ chunkIndex, count, err }, 'scheduling full rebuild');\n    scheduleFullRebuild();\n  },\n});","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Degrade-don't-fail per chunk; observe and repair\nconst importers = await collectImporters(dbPath, targets, {\n  onChunkFailure: (chunkIndex, count, err) => {\n    degradedChunks.push({ chunkIndex, count, err: String(err) });\n  },\n});\nif (degradedChunks.length > 0) queueFullRebuild(); // stale edges until then","preventionTips":["Always wire onChunkFailure in orchestration code — it is the structured signal behind the warn.","Schedule a periodic full rebuild as a safety net for degraded incremental runs.","Avoid concurrent heavy reads during incremental analyze to prevent chunk query failures."],"tags":["ladybugdb","incremental","import-graph","degraded","stale-edges"],"backgroundTag":"incremental-update-degraded","analyzedSha":"aac7515d2a8c50a1f8f923c6fb77218b333560d6","analyzedAt":"2026-08-20T23:29:22.980Z","schemaVersion":2},"datasetVersion":"2026-08-31T04:17:50.494Z"}