mastra-ai/mastra · critical · Error

Dataset item identity history is corrupt for externalId: ${f

Error message

Dataset item identity history is corrupt for externalId: ${first.externalId}

What it means

A consistency invariant check inside batchInsertItems: two different item ids were found with the same first (oldest) SCD-2 history row for one externalId. The internal identity history is corrupted and the store refuses to guess which lineage to extend.

Source

Thrown at packages/core/src/storage/domains/datasets/inmemory.ts:560

    };
  }

  // --- Bulk operations (SCD-2 internally) ---

  protected async _doBatchInsertItems(input: BatchInsertItemsInput): Promise<DatasetItem[]> {
    const dataset = this.db.datasets.get(input.datasetId);
    if (!dataset) {
      throw new Error(`Dataset not found: ${input.datasetId}`);
    }
    if (input.items.length === 0) return [];

    const acceptedByExternalId = new Map<string, { first: DatasetItemRow; current: DatasetItemRow | null }>();
    for (const rows of this.db.datasetItems.values()) {
      const first = rows[0];
      if (!first || first.datasetId !== input.datasetId || !first.externalId) continue;
      const existing = acceptedByExternalId.get(first.externalId);
      if (existing && existing.first.id !== first.id) {
        throw new Error(`Dataset item identity history is corrupt for externalId: ${first.externalId}`);
      }
      acceptedByExternalId.set(first.externalId, {
        first,
        current: rows.find(row => row.validTo === null && !row.isDeleted) ?? null,
      });
    }

    const conflicts = [];
    const planned = new Map<string, { id: string; item: (typeof input.items)[number] }>();
    const plannedByExternalId = new Map<string, { id: string; item: (typeof input.items)[number] }>();
    const resolvedIds: string[] = [];

    for (const [index, item] of input.items.entries()) {
      if (!item.externalId) {
        const id = crypto.randomUUID();
        planned.set(id, { id, item });
        resolvedIds.push(id);
        continue;

View on GitHub (pinned to 75dd419e61)

Solutions

1. Do not mutate the store's internal db directly; only use public insert/update/delete APIs.
2. Rebuild the store (fresh instance) and re-insert items so a single lineage per externalId exists.
3. Find the code path that wrote duplicate lineages for the externalId and fix it.
4. Ensure externalIds are unique per dataset before batch inserting.

// Ensure externalIds are unique per dataset before batch insert
const seen = new Set<string>();
for (const item of items) {
  if (item.externalId && seen.has(item.externalId)) {
    throw new Error(`Duplicate externalId in batch: ${item.externalId}`);
  }
  seen.add(item.externalId);
}

null

try {
  await storage.datasets.batchInsertItems({ datasetId, items });
} catch (e) {
  if (String(e).includes('identity history is corrupt')) {
    // rebuild store: recreate dataset and re-insert items
  } else throw e;
}

When it happens

Trigger: Manual mutation of this.db.datasetItems (internal map) by test code; a prior bug or out-of-band write created duplicate history lineages for the same externalId; restoring/partially overwriting the in-memory db from a snapshot.

Common situations: Custom tests poking at private state; monkey-patched or subclassed stores that bypass insert paths; corrupted snapshots restored into a fresh in-memory instance.

Related errors

• Item ${id} does not belong to dataset ${datasetId}
• Dataset not found: ${datasetId}
• Dataset not found: ${input.datasetId}
• DATASETS_STORAGE_NOT_CONFIGURED
• DATASETS_STORE_NOT_AVAILABLE

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/f74f75d311c8b6ea. Report an issue: GitHub.