thedotmack/claude-mem · warning

Skipped unformattable row during backfill

Error message

Skipped unformattable row during backfill

What it means

During backfillKind, a single row that cannot be formatted into documents (e.g. malformed JSON column) is skipped rather than aborting the whole backfill. The row id is marked pending so a later run re-fetches it, self-healing once the data is fixed.

Solutions

1. Find the offending rowId in the warning and inspect/fix its column data in SQLite.
2. Rely on the pending mechanism: fix the data and the next backfill re-syncs it.
3. Fix the formatting function to tolerate the legacy shape if many rows share it.
4. Add a data migration to normalize old-format rows.

// pre-validate rows before backfill
const bad = rows.filter((r) => !isFormattable(r));
if (bad.length) console.warn(`backfill will skip rows: ${bad.map((r) => r.id).join(',')}`);

const isFormattable = (row: Row): boolean =>
  typeof row.id === 'number' && typeof row.content === 'string' && safeJsonFields(row);

try { formatRow(row); }
catch (e) {
  ChromaSyncState.markPending(project, kind, [row.id]);
  logger.warn('skipped unformattable row', { rowId: row.id }, e);
}

When it happens

Trigger: formatDocument/row mapping throws for one row — malformed JSON in a column, null in a NOT-NULL-expected field, unexpected row shape from an older schema.

Common situations: Schema migrations leaving legacy rows, hand-edited databases, partially written rows from a crashed writer.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Backfill failed
• Chroma unavailable before write; leaving documents unsynced
• Backfill marker read failed; treating backfill as complete…
• canonical content
• cloud sync ack conflict for

AI-assisted analysis of thedotmack/claude-mem@d8bc9755e7 (2026-09-17). Data as JSON: /api/errors/3a2712c868fdf449. Report an issue: GitHub.

Appendix: source

Thrown at src/services/sync/ChromaSync.ts:831

   */
  private async backfillKind<T extends { id: number }>(
    rows: T[],
    formatDocs: (row: T) => ChromaDocument[],
    kind: 'observations' | 'summaries' | 'prompts',
    backfillProject: string
  ): Promise<number> {
    const rowsWithDocs: Array<{ row: T; docs: ChromaDocument[] }> = [];
    for (const row of rows) {
      try {
        rowsWithDocs.push({ row, docs: formatDocs(row) });
      } catch (error) {
        // A single unformattable row (e.g. a malformed JSON column) must not
        // abort the whole backfill. Skip and log it so the rest of the run
        // proceeds. Mark the id pending so a later higher-id row that advances
        // the watermark does not strand it: the pending path re-fetches it on
        // the next run, which self-heals once its format issue is resolved.
        ChromaSyncState.markPending(backfillProject, kind, [row.id]);
        logger.warn('CHROMA_SYNC', 'Skipped unformattable row during backfill', {
          project: backfillProject,
          kind,
          rowId: row.id
        }, error instanceof Error ? error : new Error(String(error)));
      }
    }
    const totalDocs = rowsWithDocs.reduce((sum, { docs }) => sum + docs.length, 0);
    let processedDocs = 0;
    let consecutiveFailures = 0;

    for (let rowIndex = 0; rowIndex < rowsWithDocs.length; rowIndex += 1) {
      const { row, docs } = rowsWithDocs[rowIndex];
      if (docs.length === 0) {
        continue;
      }

      let rowComplete = true;
      for (let i = 0; i < docs.length; i += this.BATCH_SIZE) {

View on GitHub (pinned to d8bc9755e7)