mastra-ai/mastra · error · MastraError

RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_STEP_RESULT

RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_STEP_RESULT

Error message

Failed to run experiment: Error running scorer ${scorer.id} on step ${stepId}

What it means

Thrown when a scorer fails while scoring an individual workflow step result during a workflow experiment run. For each step in the workflow result, scorers run per stepId; any throw is wrapped with id RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_STEP_RESULT and the message includes both scorer id and step id.

Source

Thrown at packages/core/src/evals/run/index.ts:1245

        // TODO : Ideally this would run on the trace.WORKFLOW_STEP span...
        // then we could directly add the score to that span
        if (stepResult?.status === 'success' && stepResult.output !== undefined) {
          const stepResults: Record<string, any> = {};
          for (const scorer of stepScorers) {
            try {
              const score = await scorer.run({
                input: stepResult.payload !== undefined ? stepResult.payload : targetResult.scoringData.input,
                output: stepResult.output,
                groundTruth: item.groundTruth,
                requestContext: item.requestContext,
                scoreSource: 'experiment',
                targetScope: 'span',
                targetEntityType: EntityType.WORKFLOW_STEP,
                targetTraceId,
              });
              stepResults[scorer.id] = score;
            } catch (error) {
              throw new MastraError(
                {
                  domain: 'SCORER',
                  id: 'RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_STEP_RESULT',
                  category: 'USER',
                  text: `Failed to run experiment: Error running scorer ${scorer.id} on step ${stepId}`,
                  details: {
                    scorerId: scorer.id,
                    stepId,
                  },
                },
                error,
              );
            }
          }
          if (Object.keys(stepResults).length > 0) {
            stepScorerResults[stepId] = stepResults;
          }
        }

View on GitHub (pinned to 75dd419e61)

Solutions

Look at the cause and the named stepId to find which step/scorer combination failed.
Check that step's StepResult — if the step itself failed, fix the workflow step first.
Ensure the scorer's extract/process handles the step's actual output schema.
Confirm the stepId in scorer configuration matches the workflow's current step ids.
Skip or guard scoring for failed steps in custom scorer code.

Example fix

// before: scorer assumes step output always exists
extract: ({ run }) => run.output.text
// after
extract: ({ run }) => run.output?.text ?? ''

Defensive patterns

Strategy: type-guard

Validate before calling

// check step results are scoreable before scoring steps
for (const [stepId, step] of Object.entries(result.steps)) {
  if (step.status !== 'success') console.warn(`Skipping scoring for failed step ${stepId}`);
}

Type guard

function isScoreableStepResult(step) {
  return step != null && step.status === 'success' && step.output != null;
}

Try / catch

try {
  await experiment.run();
} catch (e) {
  if (e?.id === 'RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_STEP_RESULT') {
    console.error('Per-step scoring failed:', e.message, e.cause);
  } else throw e;
}

Prevention

Fix failing workflow steps first — scorers run on their outputs
Keep scorer per-step config step ids in sync with the workflow definition
Make step scorers tolerant of optional/missing output fields
Type step outputs with Zod schemas so mismatches surface early

When it happens

Trigger: Experiment on a Workflow target with scorers defined per-step; scorer.run is called with targetScope 'span' and EntityType.WORKFLOW_STEP for stepId and throws — e.g. the step's output doesn't match what the scorer extracts, or the step failed (status error) leaving no output to score.

Common situations: Scoring a step whose StepResult.status is 'failed' so output is undefined; scorer extract assumes fields the step schema doesn't produce; step id renamed in the workflow while scorer config still references the old id; LLM judge failure.

Related errors

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/5b29abef5d1fd715. Report an issue: GitHub.