{"record":{"id":"2774a8c959dedcde","repo":"mastra-ai/mastra","slug":"agent-response-is-required-for-prompt-alignment-sc","errorCode":null,"errorMessage":"Agent response is required for prompt alignment scoring","messagePattern":"Agent response is required for prompt alignment scoring","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/evals/src/scorers/llm/prompt-alignment/index.ts","lineNumber":129,"sourceCode":"      description: 'Analyze prompt-response alignment across multiple dimensions',\n      outputSchema: analyzeOutputSchema,\n      createPrompt: ({ run }) => {\n        const userPrompt = getUserMessageFromRunInput(run.input) ?? '';\n        const systemPrompt = getCombinedSystemPrompt(run.input) ?? '';\n        const agentResponse = getAssistantMessageFromRunOutput(run.output) ?? '';\n\n        // Validation based on evaluation mode\n        if (evaluationMode === 'user' && !userPrompt) {\n          throw new Error('User prompt is required for user prompt alignment scoring');\n        }\n        if (evaluationMode === 'system' && !systemPrompt) {\n          throw new Error('System prompt is required for system prompt alignment scoring');\n        }\n        if (evaluationMode === 'both' && !userPrompt && !systemPrompt) {\n          throw new Error('A user or system prompt is required for combined alignment scoring');\n        }\n        if (!agentResponse) {\n          throw new Error('Agent response is required for prompt alignment scoring');\n        }\n\n        return createAnalyzePrompt({\n          userPrompt,\n          systemPrompt,\n          agentResponse,\n          evaluationMode,\n          conversationHistory: historyOptions && getConversationHistoryFromRunInput(run.input, historyOptions),\n        });\n      },\n    })\n    .generateScore(({ results }) => {\n      const analysis = results.analyzeStepResult;\n\n      if (!analysis) {\n        // Default to 0 if analysis failed\n        return 0;\n      }","sourceCodeStart":111,"sourceCodeEnd":147,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/evals/src/scorers/llm/prompt-alignment/index.ts#L111-L147","documentation":"The prompt-alignment LLM scorer requires an agentResponse to analyze, because the scorer compares the user/system prompt against what the agent actually produced. During scorer creation (createPromptAlignmentScorerLLM), if evaluationMode is 'user', 'system', or 'both' but agentResponse is missing, the factory throws immediately instead of silently producing a meaningless score. It is a fail-fast guard so misconfigured scorers never run.","triggerScenarios":"Calling createPromptAlignmentScorerLLM (or the scorer factory) with no agentResponse value, e.g. agentResponse omitted, undefined, or an empty string, after the prompt checks pass (evaluationMode 'user' has userPrompt, 'system' has systemPrompt, or 'both' has at least one prompt).","commonSituations":"Wiring the scorer into an eval where the run output field mapping is wrong so agentResponse ends up undefined; building the scorer before the agent has run; renaming a variable and forgetting to pass the response; constructing the scorer in a dry-run/test harness that only supplies prompts.","solutions":["Pass the agent's response string when creating the scorer: agentResponse: result.text (or your stored response variable).","If the response comes from a run, extract it from the run output before calling the scorer and assert it is a non-empty string.","Check your variable mapping/renaming — a typo like 'reponse' silently yields undefined and trips this check."],"exampleFix":"// before\nconst scorer = createPromptAlignmentScorerLLM({\n  model: 'openai/gpt-4o',\n  evaluationMode: 'both',\n  userPrompt,\n  systemPrompt,\n});\n// after\nconst scorer = createPromptAlignmentScorerLLM({\n  model: 'openai/gpt-4o',\n  evaluationMode: 'both',\n  userPrompt,\n  systemPrompt,\n  agentResponse: await agent.generate(userPrompt).then((r) => r.text),\n});","handlingStrategy":"validation","validationCode":"function assertScorerInputs(opts: { agentResponse?: string | null }) {\n  if (!opts.agentResponse || opts.agentResponse.trim().length === 0) {\n    throw new Error('createPromptAlignmentScorerLLM requires a non-empty agentResponse');\n  }\n}","typeGuard":"const hasAgentResponse = (r: unknown): r is string => typeof r === 'string' && r.trim().length > 0;","tryCatchPattern":"try {\n  const scorer = createPromptAlignmentScorerLLM({ ...opts, agentResponse });\n} catch (err) {\n  if (err instanceof Error && err.message.includes('Agent response is required')) {\n    // log config problem and skip scorer creation\n  } else throw err;\n}","preventionTips":["Extract agentResponse from the run output immediately after generation and pass it in the same call site.","Add a unit test asserting the scorer factory receives a non-empty response string.","Use TypeScript non-optional typed inputs so omission is a compile error."],"tags":["validation","scorer","missing-argument","llm-evals"],"backgroundTag":"missing-required-argument","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}