mastra-ai/mastra · error · Error

Analysis step failed to produce results for reason generatio

Error message

Analysis step failed to produce results for reason generation

What it means

The reason-generation step of the noise-sensitivity scorer re-reads results.analyzeStepResult to build a human-readable explanation. If the analysis result is absent at this point (same root causes as the score-time check), the reason prompt cannot be built and this variant of the error is thrown.

Source

Thrown at packages/evals/src/scorers/llm/noise-sensitivity/index.ts:173

          finalScore = Math.min(finalScore, calculatedScore);
        }
      }

      // Apply penalty for major issues
      const majorIssues = analysisResult.majorIssues || [];
      const issuesPenalty = Math.min(majorIssues.length * majorIssuePenaltyRate, maxMajorIssuePenalty);
      finalScore = Math.max(0, finalScore - issuesPenalty);

      return roundToTwoDecimals(finalScore);
    })
    .generateReason({
      description: 'Generate human-readable explanation of noise sensitivity evaluation',
      createPrompt: ({ run, results, score }) => {
        const originalQuery = getUserMessageFromRunInput(run.input) ?? '';
        const analysisResult = results.analyzeStepResult;

        if (!analysisResult) {
          throw new Error('Analysis step failed to produce results for reason generation');
        }

        return createReasonPrompt({
          userQuery: originalQuery,
          score,
          dimensions: analysisResult.dimensions || [],
          majorIssues: analysisResult.majorIssues || [],
          overallAssessment: analysisResult.overallAssessment,
        });
      },
    });
}

View on GitHub (pinned to 75dd419e61)

Solutions

1. Fix the analyze step so it always produces analyzeStepResult (capable judge model, valid schema output)
2. Retry the whole scorer run on transient LLM failures rather than partial steps
3. Check judge-model logs for refusals, truncation, or malformed JSON around the analyze call
4. Upgrade @mastra/evals if a fixed version hardened the analyze step's output parsing

Example fix

// before
const scorer = createNoiseSensitivityScorerLLM({ model: weakModel, options });
// after
const scorer = createNoiseSensitivityScorerLLM({ model: 'openai/gpt-4o', options }); // judge model that reliably emits the schema

// Same pre-condition as the analyze step; verify before running the scorer
if (!isJudgeModelCapableOfSchemaOutput(model)) {
  throw new Error('Choose a judge model that reliably emits the analyze schema');
}

function hasAnalysisForReason(results) {
  return Array.isArray(results?.analyzeStepResult?.dimensions);
}

try {
  const result = await scorer.run({ input, output });
} catch (e) {
  if (e.message.includes('reason generation')) {
    console.error('Analyze step produced no result; check judge model output', e);
    return null; // skip reason generation for this run
  }
  throw e;
}

When it happens

Trigger: The reason step runs when the analyze step result is missing from results — e.g. analyze output failed schema validation or the step silently skipped, then reason generation executes against undefined.

Common situations: Same as the score-time missing-analysis case: judge model failure/refusal, invalid JSON output, rate limits — with the additional wrinkle that the reason step may surface the failure even in configurations where scoring is short-circuited differently.

Related errors

• Analysis step failed to produce results
• MASTR_SCORER_FAILED_TO_RUN_WORKFLOW_FAILED
• Structured output returned no object
• No context available for evaluation
• Both original query and noisy response are required for eval

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/8a625876d6975b9f. Report an issue: GitHub.