mastra-ai/mastra · error · Error

Input and output messages cannot be null or empty

Error message

Input and output messages cannot be null or empty

What it means

The tool-call accuracy scorer's preprocess step validates the run shape before scoring: run.input must exist with a non-empty inputMessages array, and run.output must be a non-empty array. If either is missing or empty there are no messages to extract tool calls from, so it throws rather than silently scoring zero. Unlike 2112 this fires at scoring time (inside the scorer's preprocess), reflecting malformed run data rather than factory config.

Source

Thrown at packages/evals/src/scorers/code/tool-call-accuracy/index.ts:88

  const getDescription = () => {
    return expectedToolOrder
      ? `Evaluates whether the LLM called tools in the correct order: [${expectedToolOrder.join(', ')}]`
      : `Evaluates whether the LLM selected the correct tool (${expectedTool}) from the available tools`;
  };

  return createScorer({
    id: 'code-tool-call-accuracy-scorer',
    name: 'Tool Call Accuracy Scorer',
    description: getDescription(),
    type: 'agent',
  })
    .preprocess(async ({ run }) => {
      const isInputInvalid = !run.input || !run.input.inputMessages || run.input.inputMessages.length === 0;
      const isOutputInvalid = !run.output || run.output.length === 0;

      if (isInputInvalid || isOutputInvalid) {
        throw new Error('Input and output messages cannot be null or empty');
      }

      const { tools: actualTools, toolCallInfos } = extractToolCalls(run.output);

      const correctToolCalled = expectedTool
        ? strictMode
          ? actualTools.length === 1 && actualTools[0] === expectedTool
          : actualTools.includes(expectedTool)
        : false;

      return {
        expectedTool,
        actualTools,
        strictMode,
        expectedToolOrder,
        hasToolCalls: actualTools.length > 0,
        correctToolCalled,
        toolCallInfos,

View on GitHub (pinned to 75dd419e61)

Solutions

1. Ensure the run passed to run() includes input: { inputMessages: [...] } with at least one message
2. Ensure output is a non-empty array containing the agent's messages (including tool calls)
3. Validate the run object shape before invoking the scorer, and skip/flag runs with no output as errored rather than scored
4. Check that you are passing a ScorerRunInputForLLMJudge-shaped object, not raw messages

Example fix

// before
await scorer.run({ input: { inputMessages: [] }, output: [] });

// after
if (!run.input?.inputMessages?.length || !run.output?.length) {
  console.warn('skipping scorer: empty run');
} else {
  await scorer.run(run);
}

const isScorable = (run: { input?: { inputMessages?: unknown[] }; output?: unknown[] }) =>
  Boolean(run.input?.inputMessages?.length) && Boolean(run.output?.length);
if (!isScorable(runPayload)) skipScoring(runPayload);

function isScorableRun(r: unknown): r is { input: { inputMessages: unknown[] }; output: unknown[] } {
  const o = r as any;
  return !!o?.input?.inputMessages?.length && Array.isArray(o.output) && o.output.length > 0;
}

try {
  await scorer.run(payload);
} catch (err) {
  if ((err as Error).message === 'Input and output messages cannot be null or empty') {
    markRunErrored(payload, 'empty run payload');
    return;
  }
  throw err;
}

When it happens

Trigger: Calling scorer.run({ input: undefined, output: [...] }) or with run.input.inputMessages = []; passing an empty output array; a run object whose fields were renamed/misnested (e.g. passing messages at the wrong key).

Common situations: Unit tests constructing minimal run payloads that omit inputMessages; streaming runs that produced zero output messages (agent crashed early); wrapping another scorer and forwarding the wrong object shape.

Related errors

• Either expectedTool or expectedToolOrder must be provided
• MASTR_SCORER_FAILED_TO_CREATE_MISSING_ID
• MASTR_SCORER_FAILED_TO_RUN_MISSING_GENERATE_SCORE
• MASTR_SCORER_FAILED_TO_RUN_WORKFLOW_FAILED
• RUN_EXPERIMENT_FAILED_NO_DATA_PROVIDED

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/c7064f79cc516c93. Report an issue: GitHub.