{"record":{"id":"1fa3130867c318ee","repo":"mastra-ai/mastra","slug":"dataset-item-identity-history-is-corrupt-for-exter","errorCode":null,"errorMessage":"Dataset item identity history is corrupt for externalId: ${row.externalId}","messagePattern":"Dataset item identity history is corrupt for externalId: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/core/src/storage/domains/datasets/identity.ts","lineNumber":67,"sourceCode":"}\n\nexport interface DatasetItemBatchPlan {\n  inserts: Array<{ id: string; item: BatchInsertItemsInput['items'][number] }>;\n  resolvedIds: string[];\n  existingCurrentItems: Map<string, DatasetItemRow>;\n}\n\nexport function planDatasetItemBatch(\n  items: BatchInsertItemsInput['items'],\n  historyRows: DatasetItemRow[],\n  createId: () => string,\n): DatasetItemBatchPlan {\n  const accepted = new Map<string, { first: DatasetItemRow; current: DatasetItemRow | null }>();\n  for (const row of historyRows.sort((a, b) => a.datasetVersion - b.datasetVersion)) {\n    if (!row.externalId) continue;\n    const entry = accepted.get(row.externalId);\n    if (entry && entry.first.id !== row.id) {\n      throw new Error(`Dataset item identity history is corrupt for externalId: ${row.externalId}`);\n    }\n    if (!entry) accepted.set(row.externalId, { first: row, current: null });\n    if (row.validTo === null) accepted.get(row.externalId)!.current = row.isDeleted ? null : row;\n  }\n\n  const conflicts: DatasetItemIdentityConflictDetail[] = [];\n  const inserts: DatasetItemBatchPlan['inserts'] = [];\n  const resolvedIds: string[] = [];\n  const existingCurrentItems = new Map<string, DatasetItemRow>();\n  const requestLocal = new Map<string, DatasetItemBatchPlan['inserts'][number]>();\n\n  for (const [index, item] of items.entries()) {\n    if (!item.externalId) {\n      const insert = { id: createId(), item };\n      inserts.push(insert);\n      resolvedIds.push(insert.id);\n      continue;\n    }","sourceCodeStart":49,"sourceCodeEnd":85,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/core/src/storage/domains/datasets/identity.ts#L49-L85","documentation":"planDatasetItemBatch() replays an item's SCD-2 history sorted by datasetVersion and requires every historical row sharing an externalId to reference the same underlying item id. If two different item ids are found for one externalId, the identity index is inconsistent and this Error is thrown rather than silently merging histories.","triggerScenarios":"Calling batchInsertItems() with externalIds whose stored history rows contain rows with differing `id` values for the same externalId — i.e., a history table where an externalId maps to multiple physical items.","commonSituations":"Manual edits or migrations of the datasetItems table, restored/partial backups that re-created items with new ids while keeping old rows, concurrent inserts of the same externalId bypassing the planning step, or a storage adapter bug.","solutions":["Inspect the item history rows (getItemHistory / DB query) for that externalId and remove or re-link the divergent rows so one externalId maps to one item id.","Restore the table from a consistent backup instead of hand-patching rows.","Avoid direct writes to datasetItems; only mutate via the public dataset APIs which plan batches atomically.","If reproducible on a fresh install with correct API usage, file a bug with the storage adapter maintainer."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"const history = await storage.datasets.getItemHistory(itemId);\nconst ids = new Set(history.filter(r => r.externalId === externalId).map(r => r.id));\nif (ids.size > 1) throw new Error(`Corrupt history: externalId '${externalId}' maps to ${ids.size} items`);","typeGuard":"function hasConsistentIdentity(rows: { externalId: string | null; id: string }[]): boolean {\n  const byExternal = new Map<string, Set<string>>();\n  for (const r of rows) {\n    if (!r.externalId) continue;\n    const set = byExternal.get(r.externalId) ?? new Set();\n    set.add(r.id);\n    byExternal.set(r.externalId, set);\n  }\n  return [...byExternal.values()].every(s => s.size === 1);\n}","tryCatchPattern":"try {\n  await storage.datasets.batchInsertItems({ datasetId, items });\n} catch (e) {\n  if (e instanceof Error && e.message.includes('identity history is corrupt')) {\n    // halt, repair/restore the items table, do NOT retry blindly\n  } else throw e;\n}","preventionTips":["Never write to datasetItems tables directly; use the public dataset APIs only.","Validate backups/restores by running identity-consistency checks on history rows.","Avoid duplicate externalId inserts across concurrent writers on non-transactional stores."],"tags":["storage","datasets","data-corruption","scd2"],"backgroundTag":"dataset-item-identity-corrupt","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}