mastra-ai/mastra · error · HTTPException

DATASET_ITEM_IDENTITY_CONFLICT

DATASET_ITEM_IDENTITY_CONFLICT

Error message

error.message (DATASET_ITEM_IDENTITY_CONFLICT, cause: conflicts)

What it means

When ds.addItems detects that an item's identity (e.g. externalId or unique item ID) already exists in the dataset, it throws a MastraError with id DATASET_ITEM_IDENTITY_CONFLICT. The handler maps this to HTTP 409 Conflict and includes cause.conflicts listing the conflicting entries.

Source

Thrown at packages/server/src/server/handlers/datasets.ts:1223

          metadata?: Record<string, unknown>;
          source?: DatasetItemSource;
        }>;
      };
      const ds = await mastra.datasets.get({ id: datasetId });
      const addedItems = await ds.addItems({
        items: items.map(item => ({ ...item, externalId: item.externalId ?? undefined })),
      });
      return { items: addedItems, count: addedItems.length };
    } catch (error) {
      if (isSchemaValidationError(error)) {
        throw new HTTPException(400, {
          message: error.message,
          cause: { field: error.field, errors: error.errors },
        });
      }
      if (error instanceof MastraError) {
        if (error.id === 'DATASET_ITEM_IDENTITY_CONFLICT') {
          throw new HTTPException(409, {
            message: error.message,
            cause: { conflicts: 'conflicts' in error ? error.conflicts : [] },
          });
        }
        if (error.id === 'DATASET_ITEM_EXTERNAL_ID_INVALID') {
          throw new HTTPException(400, { message: error.message, cause: { field: 'externalId' } });
        }
        throw new HTTPException(getHttpStatusForMastraError(error.id) as StatusCode, { message: error.message });
      }
      return handleError(error, 'Error batch inserting items');
    }
  },
});

export const BATCH_DELETE_ITEMS_ROUTE = createRoute({
  method: 'DELETE',
  path: '/datasets/:datasetId/items/batch',
  responseType: 'json',

View on GitHub (pinned to 75dd419e61)

Solutions

1. Inspect cause.conflicts in the 409 response to identify the duplicate externalIds/IDs.
2. Deduplicate the batch payload (e.g. new Map(items.map(i => [i.externalId, i])).values()) before sending.
3. Check for existing items via the list/search items endpoint and skip or update them instead of re-inserting.
4. If re-insertion is intentional, delete the existing items first or use an update/upsert path if available.

Example fix

// before
await client.batchInsertItems(dsId, items); // 409 DATASET_ITEM_IDENTITY_CONFLICT
// after
const unique = [...new Map(items.map(i => [i.externalId ?? i.id, i])).values()];
await client.batchInsertItems(dsId, unique);

const keys = items.map(i => i.externalId ?? i.id).filter(Boolean);
if (new Set(keys).size !== keys.length) {
  throw new Error('Duplicate externalIds/ids within batch payload');
}
const existing = await fetch(`/api/datasets/${datasetId}/items?perPage=100`).then(r => r.json());
const existingKeys = new Set(existing.items.map(i => i.externalId ?? i.id));
const fresh = items.filter(i => !existingKeys.has(i.externalId ?? i.id));

function hasConflicts(body: unknown): body is { message: string; cause: { conflicts: unknown[] } } {
  return typeof body === 'object' && body !== null &&
    (body as any).cause?.conflicts !== undefined;
}

try {
  return await batchInsertItems(datasetId, items);
} catch (err) {
  if (err.status === 409 && err.body?.cause?.conflicts) {
    const conflictKeys = err.body.cause.conflicts;
    console.warn('Identity conflicts, retrying with deduplicated batch:', conflictKeys);
    return batchInsertItems(datasetId, items.filter(i => !conflictKeys.includes(i.externalId ?? i.id)));
  }
  throw err;
}

When it happens

Trigger: POST /datasets/:datasetId/items/batch where two items in the batch share an externalId, or an item's externalId/ID already exists in the dataset (and upsert semantics are not used).

Common situations: Re-running an import script that already inserted the items; duplicate externalIds inside one batch payload; two clients inserting the same external record concurrently; migrating data without deduplication.

Related errors

• Agent with id ${input.agent.id} already exists
• DATASET_ID_CONFLICT
• Skill with id "${id}" already exists.
• error.message (schema validation error, cause: field/errors)
• DATASET_ITEM_EXTERNAL_ID_INVALID

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/a5dfbb6ef800892e. Report an issue: GitHub.