mastra-ai/mastra · error

TOOL_MOCK_MISMATCH

TOOL_MOCK_MISMATCH

Error message

TOOL_MOCK_MISMATCH

What it means

TOOL_MOCK_MISMATCH is a failure code in DatasetToolMockReport. When dataset tool mocks are used, a live tool invocation arrived whose toolName/args did not match any declared mock (or matched at the wrong point), so the report records failure.code = 'TOOL_MOCK_MISMATCH' alongside the offending toolName and args.

Source

Thrown at packages/core/src/storage/types.ts:2519

  toolName: string;
  args: Record<string, unknown>;
  output: unknown;
  /** Argument matching mode. `strict` (default) deep-equals args; `ignore` matches on toolName only. */
  matchArgs?: 'strict' | 'ignore';
}

/**
 * Diagnostic receipt for tool-mock usage on a single experiment result.
 * Structurally mirrors `ToolMockReport` in the experiment engine.
 */
export type DatasetUnmockedToolPolicy = 'allow' | 'deny';

export interface DatasetToolMockReport {
  served: { mockIndex: number; toolName: string; args: unknown }[];
  unconsumed: { mockIndex: number; toolName: string; args: unknown }[];
  liveCalls: { toolName: string; args: unknown }[];
  failure?: {
    code: 'TOOL_MOCK_MISMATCH' | 'TOOL_MOCK_EXHAUSTED' | 'TOOL_MOCK_NOT_DECLARED';
    toolName: string;
    args: unknown;
  };
}

export interface DatasetItem {
  id: string;
  datasetId: string;
  datasetVersion: number;
  /** Caller-defined, dataset-local logical identity. Immutable after insertion. */
  externalId?: string | null;
  /** Inherited from the parent dataset at insert time. */
  organizationId?: string | null;
  /** Inherited from the parent dataset at insert time. */
  projectId?: string | null;
  input: unknown;
  groundTruth?: unknown;
  expectedTrajectory?: unknown;

View on GitHub (pinned to 75dd419e61)

Solutions

1. Read report.failure.toolName/args and add or adjust the mock entry to match the actual invocation.
2. Loosen the mock matcher so variable args (ids, timestamps) don't cause mismatches.
3. Update the dataset/prompt so the expected tool sequence matches the agent's behavior.

Example fix

// before
mocks: [{ toolName: 'getWeather', args: { city: 'Paris' } }]
// after (agent actually called with city: 'paris, fr')
mocks: [{ toolName: 'getWeather', args: { city: 'paris, fr' } }]

// Before the run, ensure every tool the agent may call is declared in the mock set
const declared = new Set(mocks.map(m => m.toolName));
const report = runDataset(...);
for (const call of report.liveCalls) {
  if (!declared.has(call.toolName)) console.warn(`Tool ${call.toolName} has no mock`);
}

function isMockMismatch(report: DatasetToolMockReport): boolean {
  return report.failure?.code === 'TOOL_MOCK_MISMATCH';
}

const report = await runDataset(ds);
if (report.failure?.code === 'TOOL_MOCK_MISMATCH') {
  console.error(`Unexpected call to ${report.failure.toolName}`, report.failure.args);
}

When it happens

Trigger: An agent run under a mocked dataset calls a tool that was not mocked, or calls it with different arguments than the mock declared; tool arg serialization differs (order/extra keys) from the mock's expected args.

Common situations: Evals/dataset runs where the model chooses a different tool than expected; prompts changed so the agent calls tools in a different order or with different inputs; non-deterministic args (timestamps, random ids) that never match mock args.

Related errors

• DATASETS_STORAGE_NOT_CONFIGURED
• DATASETS_STORE_NOT_AVAILABLE
• DATASET_NOT_FOUND
• DATASETS_STORAGE_NOT_CONFIGURED
• DATASETS_STORE_NOT_AVAILABLE

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/2cdd53a0fab86e9b. Report an issue: GitHub.