{"record":{"id":"8a625876d6975b9f","repo":"mastra-ai/mastra","slug":"analysis-step-failed-to-produce-results-for-reason","errorCode":null,"errorMessage":"Analysis step failed to produce results for reason generation","messagePattern":"Analysis step failed to produce results for reason generation","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/evals/src/scorers/llm/noise-sensitivity/index.ts","lineNumber":173,"sourceCode":"          finalScore = Math.min(finalScore, calculatedScore);\n        }\n      }\n\n      // Apply penalty for major issues\n      const majorIssues = analysisResult.majorIssues || [];\n      const issuesPenalty = Math.min(majorIssues.length * majorIssuePenaltyRate, maxMajorIssuePenalty);\n      finalScore = Math.max(0, finalScore - issuesPenalty);\n\n      return roundToTwoDecimals(finalScore);\n    })\n    .generateReason({\n      description: 'Generate human-readable explanation of noise sensitivity evaluation',\n      createPrompt: ({ run, results, score }) => {\n        const originalQuery = getUserMessageFromRunInput(run.input) ?? '';\n        const analysisResult = results.analyzeStepResult;\n\n        if (!analysisResult) {\n          throw new Error('Analysis step failed to produce results for reason generation');\n        }\n\n        return createReasonPrompt({\n          userQuery: originalQuery,\n          score,\n          dimensions: analysisResult.dimensions || [],\n          majorIssues: analysisResult.majorIssues || [],\n          overallAssessment: analysisResult.overallAssessment,\n        });\n      },\n    });\n}\n","sourceCodeStart":155,"sourceCodeEnd":186,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/evals/src/scorers/llm/noise-sensitivity/index.ts#L155-L186","documentation":"The reason-generation step of the noise-sensitivity scorer re-reads results.analyzeStepResult to build a human-readable explanation. If the analysis result is absent at this point (same root causes as the score-time check), the reason prompt cannot be built and this variant of the error is thrown.","triggerScenarios":"The reason step runs when the analyze step result is missing from results — e.g. analyze output failed schema validation or the step silently skipped, then reason generation executes against undefined.","commonSituations":"Same as the score-time missing-analysis case: judge model failure/refusal, invalid JSON output, rate limits — with the additional wrinkle that the reason step may surface the failure even in configurations where scoring is short-circuited differently.","solutions":["Fix the analyze step so it always produces analyzeStepResult (capable judge model, valid schema output)","Retry the whole scorer run on transient LLM failures rather than partial steps","Check judge-model logs for refusals, truncation, or malformed JSON around the analyze call","Upgrade @mastra/evals if a fixed version hardened the analyze step's output parsing"],"exampleFix":"// before\nconst scorer = createNoiseSensitivityScorerLLM({ model: weakModel, options });\n// after\nconst scorer = createNoiseSensitivityScorerLLM({ model: 'openai/gpt-4o', options }); // judge model that reliably emits the schema","handlingStrategy":"try-catch","validationCode":"// Same pre-condition as the analyze step; verify before running the scorer\nif (!isJudgeModelCapableOfSchemaOutput(model)) {\n  throw new Error('Choose a judge model that reliably emits the analyze schema');\n}","typeGuard":"function hasAnalysisForReason(results) {\n  return Array.isArray(results?.analyzeStepResult?.dimensions);\n}","tryCatchPattern":"try {\n  const result = await scorer.run({ input, output });\n} catch (e) {\n  if (e.message.includes('reason generation')) {\n    console.error('Analyze step produced no result; check judge model output', e);\n    return null; // skip reason generation for this run\n  }\n  throw e;\n}","preventionTips":["Treat the analyze-step failure as the root cause; harden it first (schema prompts, retries)","Use structured-output/JSON mode on the judge model to reduce parse failures","Add retries around the whole scorer run rather than individual steps"],"tags":["evals","llm","runtime","schema-validation"],"backgroundTag":"llm-structured-output-missing","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}