{"record":{"id":"2cdd53a0fab86e9b","repo":"mastra-ai/mastra","slug":"tool-mock-mismatch","errorCode":"TOOL_MOCK_MISMATCH","errorMessage":"TOOL_MOCK_MISMATCH","messagePattern":"TOOL_MOCK_MISMATCH","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"packages/core/src/storage/types.ts","lineNumber":2519,"sourceCode":"  toolName: string;\n  args: Record<string, unknown>;\n  output: unknown;\n  /** Argument matching mode. `strict` (default) deep-equals args; `ignore` matches on toolName only. */\n  matchArgs?: 'strict' | 'ignore';\n}\n\n/**\n * Diagnostic receipt for tool-mock usage on a single experiment result.\n * Structurally mirrors `ToolMockReport` in the experiment engine.\n */\nexport type DatasetUnmockedToolPolicy = 'allow' | 'deny';\n\nexport interface DatasetToolMockReport {\n  served: { mockIndex: number; toolName: string; args: unknown }[];\n  unconsumed: { mockIndex: number; toolName: string; args: unknown }[];\n  liveCalls: { toolName: string; args: unknown }[];\n  failure?: {\n    code: 'TOOL_MOCK_MISMATCH' | 'TOOL_MOCK_EXHAUSTED' | 'TOOL_MOCK_NOT_DECLARED';\n    toolName: string;\n    args: unknown;\n  };\n}\n\nexport interface DatasetItem {\n  id: string;\n  datasetId: string;\n  datasetVersion: number;\n  /** Caller-defined, dataset-local logical identity. Immutable after insertion. */\n  externalId?: string | null;\n  /** Inherited from the parent dataset at insert time. */\n  organizationId?: string | null;\n  /** Inherited from the parent dataset at insert time. */\n  projectId?: string | null;\n  input: unknown;\n  groundTruth?: unknown;\n  expectedTrajectory?: unknown;","sourceCodeStart":2501,"sourceCodeEnd":2537,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/core/src/storage/types.ts#L2501-L2537","documentation":"TOOL_MOCK_MISMATCH is a failure code in DatasetToolMockReport. When dataset tool mocks are used, a live tool invocation arrived whose toolName/args did not match any declared mock (or matched at the wrong point), so the report records failure.code = 'TOOL_MOCK_MISMATCH' alongside the offending toolName and args.","triggerScenarios":"An agent run under a mocked dataset calls a tool that was not mocked, or calls it with different arguments than the mock declared; tool arg serialization differs (order/extra keys) from the mock's expected args.","commonSituations":"Evals/dataset runs where the model chooses a different tool than expected; prompts changed so the agent calls tools in a different order or with different inputs; non-deterministic args (timestamps, random ids) that never match mock args.","solutions":["Read report.failure.toolName/args and add or adjust the mock entry to match the actual invocation.","Loosen the mock matcher so variable args (ids, timestamps) don't cause mismatches.","Update the dataset/prompt so the expected tool sequence matches the agent's behavior."],"exampleFix":"// before\nmocks: [{ toolName: 'getWeather', args: { city: 'Paris' } }]\n// after (agent actually called with city: 'paris, fr')\nmocks: [{ toolName: 'getWeather', args: { city: 'paris, fr' } }]","handlingStrategy":"validation","validationCode":"// Before the run, ensure every tool the agent may call is declared in the mock set\nconst declared = new Set(mocks.map(m => m.toolName));\nconst report = runDataset(...);\nfor (const call of report.liveCalls) {\n  if (!declared.has(call.toolName)) console.warn(`Tool ${call.toolName} has no mock`);\n}","typeGuard":"function isMockMismatch(report: DatasetToolMockReport): boolean {\n  return report.failure?.code === 'TOOL_MOCK_MISMATCH';\n}","tryCatchPattern":"const report = await runDataset(ds);\nif (report.failure?.code === 'TOOL_MOCK_MISMATCH') {\n  console.error(`Unexpected call to ${report.failure.toolName}`, report.failure.args);\n}","preventionTips":["Check the unconsumed and liveCalls arrays after every dataset run to spot mismatched mocks early.","Avoid volatile args (timestamps, random ids) in mock matchers or normalize them.","Keep dataset prompts and expected tool sequences in sync when changing agent instructions."],"tags":["datasets","tool-mocking","evaluation"],"backgroundTag":"tool-mock-mismatch","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}