{"record":{"id":"a73c909291c8433b","repo":"mastra-ai/mastra","slug":"input-and-output-messages-cannot-be-null-or-empty-a73c90","errorCode":null,"errorMessage":"Input and output messages cannot be null or empty","messagePattern":"Input and output messages cannot be null or empty","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/evals/src/scorers/llm/tool-call-accuracy/index.ts","lineNumber":50,"sourceCode":"export function createToolCallAccuracyScorerLLM({ model, availableTools }: ToolCallAccuracyOptions) {\n  const toolDefinitions = availableTools.map(tool => `${tool.id}: ${tool.description}`).join('\\n');\n\n  return createScorer({\n    id: 'llm-tool-call-accuracy-scorer',\n    name: 'Tool Call Accuracy (LLM)',\n    description: 'Evaluates whether an agent selected appropriate tools for the given task using LLM analysis',\n    judge: {\n      model,\n      instructions: TOOL_SELECTION_ACCURACY_INSTRUCTIONS,\n    },\n    type: 'agent',\n  })\n    .preprocess(async ({ run }) => {\n      const isInputInvalid = !run.input || !run.input.inputMessages || run.input.inputMessages.length === 0;\n      const isOutputInvalid = !run.output || run.output.length === 0;\n\n      if (isInputInvalid || isOutputInvalid) {\n        throw new Error('Input and output messages cannot be null or empty');\n      }\n\n      const { tools: actualTools, toolCallInfos } = extractToolCalls(run.output);\n\n      return {\n        actualTools,\n        hasToolCalls: actualTools.length > 0,\n        toolCallInfos,\n      };\n    })\n    .analyze({\n      description: 'Analyze the appropriateness of tool selections',\n      outputSchema: analyzeOutputSchema,\n      createPrompt: ({ run, results }) => {\n        const userInput = getUserMessageFromRunInput(run.input) ?? '';\n        const agentResponse = getAssistantMessageFromRunOutput(run.output) ?? '';\n\n        const toolsCalled = results.preprocessStepResult?.actualTools || [];","sourceCodeStart":32,"sourceCodeEnd":68,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/evals/src/scorers/llm/tool-call-accuracy/index.ts#L32-L68","documentation":"The tool-call-accuracy scorer needs both the conversation input messages (run.input.inputMessages) and the output messages to extract which tools were actually called. If either is null, undefined, or an empty array, the scorer's preprocess step throws because no accuracy comparison is possible. This is a runtime guard inside the scorer pipeline, not the constructor.","triggerScenarios":"Invoking the tool-call-accuracy scorer with a run whose run.input.inputMessages is missing/empty or whose run.output array is empty — e.g. scoring a run that produced no output, passing a malformed ScorerRunInputForLLMJudge, or fetching a run from storage before the output was persisted.","commonSituations":"Scoring partially-failed or interrupted agent runs; replaying stored traces where output arrays are empty; feeding the scorer custom input objects that omit inputMessages; filtering runs by date and including ones still in progress.","solutions":["Verify the run you pass has non-empty inputMessages and output arrays before scoring.","If the run genuinely produced no output, skip it — filter runs with output.length === 0 out of your eval set.","Check how you build run.input; inputMessages must be under run.input.inputMessages, not a sibling key."],"exampleFix":"// before\nfor (const run of runs) {\n  await scorer.run(run);\n}\n// after\nfor (const run of runs) {\n  const ok = run?.input?.inputMessages?.length && run?.output?.length;\n  if (!ok) continue;\n  await scorer.run(run);\n}","handlingStrategy":"type-guard","validationCode":"function isScorableRun(run: { input?: { inputMessages?: unknown[] }; output?: unknown[] }): boolean {\n  return Boolean(run?.input?.inputMessages?.length && run?.output?.length);\n}","typeGuard":"const isNonEmptyArray = <T>(a: T[] | null | undefined): a is T[] => Array.isArray(a) && a.length > 0;","tryCatchPattern":"try {\n  await scorer.run(run);\n} catch (err) {\n  if (err instanceof Error && err.message.includes('cannot be null or empty')) {\n    logger.warn({ runId: run.id }, 'skipping run with empty input/output');\n  } else throw err;\n}","preventionTips":["Filter eval runs for non-empty inputMessages and output before scoring.","Check run completion status before pulling runs from storage.","Build run inputs via a typed constructor so inputMessages cannot be omitted."],"tags":["validation","empty-input","scorer","preprocess"],"backgroundTag":"missing-required-argument","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T08:17:16.595Z"}