mastra-ai/mastra · error · Error

Dataset item identity history is corrupt for externalId: ${r

Error message

Dataset item identity history is corrupt for externalId: ${row.externalId}

What it means

planDatasetItemBatch() replays an item's SCD-2 history sorted by datasetVersion and requires every historical row sharing an externalId to reference the same underlying item id. If two different item ids are found for one externalId, the identity index is inconsistent and this Error is thrown rather than silently merging histories.

Source

Thrown at packages/core/src/storage/domains/datasets/identity.ts:67

}

export interface DatasetItemBatchPlan {
  inserts: Array<{ id: string; item: BatchInsertItemsInput['items'][number] }>;
  resolvedIds: string[];
  existingCurrentItems: Map<string, DatasetItemRow>;
}

export function planDatasetItemBatch(
  items: BatchInsertItemsInput['items'],
  historyRows: DatasetItemRow[],
  createId: () => string,
): DatasetItemBatchPlan {
  const accepted = new Map<string, { first: DatasetItemRow; current: DatasetItemRow | null }>();
  for (const row of historyRows.sort((a, b) => a.datasetVersion - b.datasetVersion)) {
    if (!row.externalId) continue;
    const entry = accepted.get(row.externalId);
    if (entry && entry.first.id !== row.id) {
      throw new Error(`Dataset item identity history is corrupt for externalId: ${row.externalId}`);
    }
    if (!entry) accepted.set(row.externalId, { first: row, current: null });
    if (row.validTo === null) accepted.get(row.externalId)!.current = row.isDeleted ? null : row;
  }

  const conflicts: DatasetItemIdentityConflictDetail[] = [];
  const inserts: DatasetItemBatchPlan['inserts'] = [];
  const resolvedIds: string[] = [];
  const existingCurrentItems = new Map<string, DatasetItemRow>();
  const requestLocal = new Map<string, DatasetItemBatchPlan['inserts'][number]>();

  for (const [index, item] of items.entries()) {
    if (!item.externalId) {
      const insert = { id: createId(), item };
      inserts.push(insert);
      resolvedIds.push(insert.id);
      continue;
    }

View on GitHub (pinned to 75dd419e61)

Solutions

1. Inspect the item history rows (getItemHistory / DB query) for that externalId and remove or re-link the divergent rows so one externalId maps to one item id.
2. Restore the table from a consistent backup instead of hand-patching rows.
3. Avoid direct writes to datasetItems; only mutate via the public dataset APIs which plan batches atomically.
4. If reproducible on a fresh install with correct API usage, file a bug with the storage adapter maintainer.

const history = await storage.datasets.getItemHistory(itemId);
const ids = new Set(history.filter(r => r.externalId === externalId).map(r => r.id));
if (ids.size > 1) throw new Error(`Corrupt history: externalId '${externalId}' maps to ${ids.size} items`);

function hasConsistentIdentity(rows: { externalId: string | null; id: string }[]): boolean {
  const byExternal = new Map<string, Set<string>>();
  for (const r of rows) {
    if (!r.externalId) continue;
    const set = byExternal.get(r.externalId) ?? new Set();
    set.add(r.id);
    byExternal.set(r.externalId, set);
  }
  return [...byExternal.values()].every(s => s.size === 1);
}

try {
  await storage.datasets.batchInsertItems({ datasetId, items });
} catch (e) {
  if (e instanceof Error && e.message.includes('identity history is corrupt')) {
    // halt, repair/restore the items table, do NOT retry blindly
  } else throw e;
}

When it happens

Trigger: Calling batchInsertItems() with externalIds whose stored history rows contain rows with differing `id` values for the same externalId — i.e., a history table where an externalId maps to multiple physical items.

Common situations: Manual edits or migrations of the datasetItems table, restored/partial backups that re-created items with new ids while keeping old rows, concurrent inserts of the same externalId bypassing the planning step, or a storage adapter bug.

Related errors

• DATASETS_STORAGE_NOT_CONFIGURED
• DATASETS_STORE_NOT_AVAILABLE
• DATASET_NOT_FOUND
• DATASETS_STORAGE_NOT_CONFIGURED
• DATASETS_STORE_NOT_AVAILABLE

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/1fa3130867c318ee. Report an issue: GitHub.