mastra-ai/mastra · error · MastraError

RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_TRAJECTORY

RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_TRAJECTORY

Error message

Failed to run experiment: Error running trajectory scorer ${scorer.id}

What it means

Thrown when a trajectory scorer (scoring the conversation/workflow trajectory as a whole) fails during an agent experiment run. The runner wraps the underlying scorer.run error in a MastraError with id RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_TRAJECTORY, naming the scorer id.

Source

Thrown at packages/core/src/evals/run/index.ts:1181

      const trajectory = traceTrajectory ?? (rawOutput ? extractTrajectory(rawOutput) : { steps: [] });

      for (const scorer of scorers.trajectory) {
        try {
          const score = await scorer.run({
            input: targetResult.scoringData?.input,
            output: trajectory,
            groundTruth: item.groundTruth,
            expectedTrajectory: item.expectedTrajectory,
            requestContext: item.requestContext,
            scoreSource: 'experiment',
            targetScope: 'trajectory',
            targetEntityType,
            targetTraceId,
            targetSpanId: targetResult.spanId,
          });
          trajectoryScorerResults[scorer.id] = score;
        } catch (error) {
          throw new MastraError(
            {
              domain: 'SCORER',
              id: 'RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_TRAJECTORY',
              category: 'USER',
              text: `Failed to run experiment: Error running trajectory scorer ${scorer.id}`,
              details: {
                scorerId: scorer.id,
                item: JSON.stringify(item),
              },
            },
            error,
          );
        }
      }
      if (Object.keys(trajectoryScorerResults).length > 0) {
        scorerResults.trajectory = trajectoryScorerResults;
      }
    }

View on GitHub (pinned to 75dd419e61)

Solutions

Check the wrapped cause for the scorer-internal error.
Confirm the dataset items produce non-empty, well-formed trajectories (turns/inputs arrays).
Validate the trajectory scorer's expected input shape (messages, tool calls) against what the agent produced.
Test the trajectory scorer standalone on a recorded trace.
Fix judge model credentials/rate limits if LLM-based.

Example fix

// before: trajectory scorer assumes toolCalls exist
const score = trajectory.filter(t => t.toolCalls.length > 0)...
// after: guard for missing data
const score = trajectory.filter(t => (t.toolCalls ?? []).length > 0)...

Defensive patterns

Strategy: validation

Validate before calling

// ensure dataset items yield non-empty trajectories before scoring
for (const item of data) {
  const hasTurns = Array.isArray(item.turns) && item.turns.length > 0;
  const hasInputs = Array.isArray(item.inputs) && item.inputs.length > 0;
  if (!hasTurns && !hasInputs) throw new Error('Item produces an empty trajectory');
}

Type guard

function producesTrajectory(item) {
  return (Array.isArray(item.turns) && item.turns.length > 0) || (Array.isArray(item.inputs) && item.inputs.length > 0);
}

Prevention

Verify trajectory scorers expect message-shaped trajectory data, not StepResults
Test trajectory scorers on one recorded multi-turn run first
Guard trajectory processing for missing toolCalls/messages fields
Check judge model availability before long trajectory-heavy runs

When it happens

Trigger: Experiment on an Agent with scorers.trajectory configured; scorer.run is invoked with EntityType.TRAJECTORY targeting the run's trace/span and throws — e.g. the trajectory scorer cannot parse the turns/messages or its judge LLM fails.

Common situations: Multi-turn dataset items (item.turns/item.inputs) producing trajectories the scorer didn't expect (empty messages); trajectory scorer requiring tool-call data absent from the run; LLM judge misconfiguration.

Related errors

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/b380095be9fb3dae. Report an issue: GitHub.