{"record":{"id":"6c10f71954af861f","repo":"thedotmack/claude-mem","slug":"observation-watermark-bump-skipped-partial-write-row-marked","errorCode":null,"errorMessage":"Observation watermark bump skipped — partial write, row marked pending","messagePattern":"Observation watermark bump skipped — partial write, row marked pending","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/services/sync/ChromaSync.ts","lineNumber":518,"sourceCode":"      documentCount: documents.length,\n      project\n    });\n\n    // Only advance the watermark on a confirmed full write. addDocuments() now\n    // returns a written count and tolerates per-batch failures, so a transient\n    // Chroma error must NOT mark this observation as synced — otherwise the\n    // backfill pass on next boot will skip past it (CodeRabbit review on PR\n    // #2282).\n    const written = await this.addDocuments(documents);\n    if (written === documents.length) {\n      ChromaSyncState.clearPending(project, 'observations', [observationId]);\n      ChromaSyncState.bump(project, 'observations', observationId);\n    } else {\n      // Not bumping is not enough: the watermark is a high-water mark, so the\n      // next row that does write would skip past this one for good (#3917).\n      // Record it as pending so the backfill retries it, like backfillKind().\n      ChromaSyncState.markPending(project, 'observations', [observationId]);\n      logger.warn('CHROMA_SYNC', 'Observation watermark bump skipped — partial write, row marked pending', {\n        observationId,\n        project,\n        requested: documents.length,\n        written\n      });\n    }\n  }\n\n  async syncSummary(\n    summaryId: number,\n    memorySessionId: string,\n    project: string,\n    summary: ParsedSummary,\n    promptNumber: number,\n    createdAtEpoch: number,\n    platformSource?: string\n  ): Promise<void> {\n    const stored: StoredSummary = {","sourceCodeStart":500,"sourceCodeEnd":536,"githubUrl":"https://github.com/thedotmack/claude-mem/blob/d8bc9755e74915e5c3b999181e10a67c889bce2a/src/services/sync/ChromaSync.ts#L500-L536","documentation":"syncObservation writes multiple documents to Chroma; if addDocuments writes fewer than requested (partial write), the observation watermark is NOT advanced (a high-water mark would permanently skip the row) and the row is marked pending so a later backfill retries it.","triggerScenarios":"addDocuments returns written < documents.length — e.g. Chroma rejects some embeddings, a batch item fails, or the client returns partial success for the batch.","commonSituations":"Large batches hitting payload limits, intermittent Chroma client errors mid-batch, malformed document content in one row of the batch.","solutions":["Inspect why addDocuments returned fewer writes (Chroma client logs) and fix the batch error.","Rely on the pending mark: the next backfill re-syncs this observation automatically.","Reduce batch size for syncObservation to lower partial-failure likelihood.","Verify Chroma collection health/connectivity if partials recur."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"const written = await this.addDocuments(documents);\nif (written !== documents.length) console.warn(`partial write ${written}/${documents.length}; expect pending re-sync`);","typeGuard":null,"tryCatchPattern":"try { await syncObservation(...); }\ncatch (e) { /* row remains pending; backfill retries — safe to ignore transient errors */ }","preventionTips":["Keep observation batch sizes small","Monitor Chroma client errors for recurring partial failures","Ensure pending rows are periodically re-scanned by backfill","Validate document content before batching"],"tags":["sync","watermark","partial-write","chroma"],"backgroundTag":"database-write-failed","analyzedSha":"d8bc9755e74915e5c3b999181e10a67c889bce2a","analyzedAt":"2026-09-17T16:40:26.182Z","contentChangedAt":"2026-09-17T16:40:26.182Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}