{"record":{"id":"c7064f79cc516c93","repo":"mastra-ai/mastra","slug":"input-and-output-messages-cannot-be-null-or-empty","errorCode":null,"errorMessage":"Input and output messages cannot be null or empty","messagePattern":"Input and output messages cannot be null or empty","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/evals/src/scorers/code/tool-call-accuracy/index.ts","lineNumber":88,"sourceCode":"\n  const getDescription = () => {\n    return expectedToolOrder\n      ? `Evaluates whether the LLM called tools in the correct order: [${expectedToolOrder.join(', ')}]`\n      : `Evaluates whether the LLM selected the correct tool (${expectedTool}) from the available tools`;\n  };\n\n  return createScorer({\n    id: 'code-tool-call-accuracy-scorer',\n    name: 'Tool Call Accuracy Scorer',\n    description: getDescription(),\n    type: 'agent',\n  })\n    .preprocess(async ({ run }) => {\n      const isInputInvalid = !run.input || !run.input.inputMessages || run.input.inputMessages.length === 0;\n      const isOutputInvalid = !run.output || run.output.length === 0;\n\n      if (isInputInvalid || isOutputInvalid) {\n        throw new Error('Input and output messages cannot be null or empty');\n      }\n\n      const { tools: actualTools, toolCallInfos } = extractToolCalls(run.output);\n\n      const correctToolCalled = expectedTool\n        ? strictMode\n          ? actualTools.length === 1 && actualTools[0] === expectedTool\n          : actualTools.includes(expectedTool)\n        : false;\n\n      return {\n        expectedTool,\n        actualTools,\n        strictMode,\n        expectedToolOrder,\n        hasToolCalls: actualTools.length > 0,\n        correctToolCalled,\n        toolCallInfos,","sourceCodeStart":70,"sourceCodeEnd":106,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/evals/src/scorers/code/tool-call-accuracy/index.ts#L70-L106","documentation":"The tool-call accuracy scorer's preprocess step validates the run shape before scoring: run.input must exist with a non-empty inputMessages array, and run.output must be a non-empty array. If either is missing or empty there are no messages to extract tool calls from, so it throws rather than silently scoring zero. Unlike 2112 this fires at scoring time (inside the scorer's preprocess), reflecting malformed run data rather than factory config.","triggerScenarios":"Calling scorer.run({ input: undefined, output: [...] }) or with run.input.inputMessages = []; passing an empty output array; a run object whose fields were renamed/misnested (e.g. passing messages at the wrong key).","commonSituations":"Unit tests constructing minimal run payloads that omit inputMessages; streaming runs that produced zero output messages (agent crashed early); wrapping another scorer and forwarding the wrong object shape.","solutions":["Ensure the run passed to run() includes input: { inputMessages: [...] } with at least one message","Ensure output is a non-empty array containing the agent's messages (including tool calls)","Validate the run object shape before invoking the scorer, and skip/flag runs with no output as errored rather than scored","Check that you are passing a ScorerRunInputForLLMJudge-shaped object, not raw messages"],"exampleFix":"// before\nawait scorer.run({ input: { inputMessages: [] }, output: [] });\n\n// after\nif (!run.input?.inputMessages?.length || !run.output?.length) {\n  console.warn('skipping scorer: empty run');\n} else {\n  await scorer.run(run);\n}","handlingStrategy":"validation","validationCode":"const isScorable = (run: { input?: { inputMessages?: unknown[] }; output?: unknown[] }) =>\n  Boolean(run.input?.inputMessages?.length) && Boolean(run.output?.length);\nif (!isScorable(runPayload)) skipScoring(runPayload);","typeGuard":"function isScorableRun(r: unknown): r is { input: { inputMessages: unknown[] }; output: unknown[] } {\n  const o = r as any;\n  return !!o?.input?.inputMessages?.length && Array.isArray(o.output) && o.output.length > 0;\n}","tryCatchPattern":"try {\n  await scorer.run(payload);\n} catch (err) {\n  if ((err as Error).message === 'Input and output messages cannot be null or empty') {\n    markRunErrored(payload, 'empty run payload');\n    return;\n  }\n  throw err;\n}","preventionTips":["Validate run shape ({ input: { inputMessages }, output }) before calling run()","Treat zero-output runs (agent crashed early) as errored, not scored","Build run payloads from typed helpers rather than ad-hoc objects","Check field nesting: inputMessages lives under run.input"],"tags":["evals","scorer","tool-call-accuracy","empty-input"],"backgroundTag":"empty-run-payload","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}