{"record":{"id":"f716e4ef77d52649","repo":"mastra-ai/mastra","slug":"either-expectedtool-or-expectedtoolorder-must-be-p","errorCode":null,"errorMessage":"Either expectedTool or expectedToolOrder must be provided","messagePattern":"Either expectedTool or expectedToolOrder must be provided","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/evals/src/scorers/code/tool-call-accuracy/index.ts","lineNumber":68,"sourceCode":"    return checkToolOrder(actualTools, expectedToolOrder, strictMode) ? 1 : 0;\n  }\n\n  if (!expectedTool) {\n    return 0;\n  }\n\n  if (strictMode) {\n    return actualTools.length === 1 && actualTools[0] === expectedTool ? 1 : 0;\n  }\n\n  return actualTools.includes(expectedTool) ? 1 : 0;\n}\n\nexport function createToolCallAccuracyScorerCode(options: ToolCallAccuracyOptions) {\n  const { expectedTool, strictMode = false, expectedToolOrder } = options;\n\n  if (!expectedTool && !expectedToolOrder) {\n    throw new Error('Either expectedTool or expectedToolOrder must be provided');\n  }\n\n  const getDescription = () => {\n    return expectedToolOrder\n      ? `Evaluates whether the LLM called tools in the correct order: [${expectedToolOrder.join(', ')}]`\n      : `Evaluates whether the LLM selected the correct tool (${expectedTool}) from the available tools`;\n  };\n\n  return createScorer({\n    id: 'code-tool-call-accuracy-scorer',\n    name: 'Tool Call Accuracy Scorer',\n    description: getDescription(),\n    type: 'agent',\n  })\n    .preprocess(async ({ run }) => {\n      const isInputInvalid = !run.input || !run.input.inputMessages || run.input.inputMessages.length === 0;\n      const isOutputInvalid = !run.output || run.output.length === 0;\n","sourceCodeStart":50,"sourceCodeEnd":86,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/evals/src/scorers/code/tool-call-accuracy/index.ts#L50-L86","documentation":"createToolCallAccuracyScorerCode requires configuration describing what to expect: either a single expectedTool name or an expectedToolOrder array. If neither option is provided there is nothing to compare the run's tool calls against, so the factory throws at construction time instead of producing a scorer that can never pass. This is a factory-time configuration validation, not a runtime scoring failure.","triggerScenarios":"Calling createToolCallAccuracyScorerCode({}) or createToolCallAccuracyScorerCode({ strictMode: true }) without expectedTool or expectedToolOrder; building the scorer programmatically from config where both fields are undefined.","commonSituations":"Typo in the options key (expectedTools vs expectedTool); loading scorer options from JSON/env where the fields are omitted; copying a scorer factory call and deleting the expectedTool line.","solutions":["Pass expectedTool: 'toolName' when you only care whether a specific tool was called","Pass expectedToolOrder: ['toolA','toolB'] when the sequence of tool calls matters","Verify the option keys are spelled expectedTool and expectedToolOrder exactly","If you want appropriateness without a fixed expectation, use the appropriate scorer variant (toolCallAppropriatenessScorer defaults) instead"],"exampleFix":"// before\nconst scorer = createToolCallAccuracyScorerCode({});\n\n// after\nconst scorer = createToolCallAccuracyScorerCode({ expectedTool: 'getWeather' });\n// or\nconst scorer = createToolCallAccuracyScorerCode({ expectedToolOrder: ['search', 'summarize'] });","handlingStrategy":"validation","validationCode":"function validateToolAccuracyOptions(o: ToolCallAccuracyOptions): void {\n  if (!o.expectedTool && !o.expectedToolOrder) {\n    throw new Error('Provide expectedTool or expectedToolOrder');\n  }\n}\nvalidateToolAccuracyOptions(options);","typeGuard":"function hasExpectation(o: ToolCallAccuracyOptions): o is ToolCallAccuracyOptions & ({ expectedTool: string } | { expectedToolOrder: string[] }) {\n  return Boolean(o.expectedTool) || Boolean(o.expectedToolOrder);\n}","tryCatchPattern":"try {\n  const scorer = createToolCallAccuracyScorerCode(options);\n} catch (err) {\n  if ((err as Error).message.includes('expectedTool or expectedToolOrder')) {\n    throw new ConfigError('Tool-call accuracy scorer requires expectedTool or expectedToolOrder');\n  }\n  throw err;\n}","preventionTips":["Always pass expectedTool or expectedToolOrder when constructing the scorer","Check option key spelling (expectedTool, not expectedTools)","Load options from a validated schema (zod) when sourcing from config/JSON","Use toolCallAppropriatenessScorer if you do not have a fixed expectation"],"tags":["evals","scorer","tool-call-accuracy","config"],"backgroundTag":"missing-required-option","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}