{"record":{"id":"dda31abc4965cd72","repo":"mastra-ai/mastra","slug":"user-prompt-is-required-for-user-prompt-alignment","errorCode":null,"errorMessage":"User prompt is required for user prompt alignment scoring","messagePattern":"User prompt is required for user prompt alignment scoring","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/evals/src/scorers/llm/prompt-alignment/index.ts","lineNumber":120,"sourceCode":"    id: 'prompt-alignment-scorer',\n    name: 'Prompt Alignment (LLM)',\n    description: 'Evaluates how well the agent response aligns with the intent and requirements of the user prompt',\n    judge: {\n      model,\n      instructions: PROMPT_ALIGNMENT_INSTRUCTIONS,\n    },\n  })\n    .analyze({\n      description: 'Analyze prompt-response alignment across multiple dimensions',\n      outputSchema: analyzeOutputSchema,\n      createPrompt: ({ run }) => {\n        const userPrompt = getUserMessageFromRunInput(run.input) ?? '';\n        const systemPrompt = getCombinedSystemPrompt(run.input) ?? '';\n        const agentResponse = getAssistantMessageFromRunOutput(run.output) ?? '';\n\n        // Validation based on evaluation mode\n        if (evaluationMode === 'user' && !userPrompt) {\n          throw new Error('User prompt is required for user prompt alignment scoring');\n        }\n        if (evaluationMode === 'system' && !systemPrompt) {\n          throw new Error('System prompt is required for system prompt alignment scoring');\n        }\n        if (evaluationMode === 'both' && !userPrompt && !systemPrompt) {\n          throw new Error('A user or system prompt is required for combined alignment scoring');\n        }\n        if (!agentResponse) {\n          throw new Error('Agent response is required for prompt alignment scoring');\n        }\n\n        return createAnalyzePrompt({\n          userPrompt,\n          systemPrompt,\n          agentResponse,\n          evaluationMode,\n          conversationHistory: historyOptions && getConversationHistoryFromRunInput(run.input, historyOptions),\n        });","sourceCodeStart":102,"sourceCodeEnd":138,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/evals/src/scorers/llm/prompt-alignment/index.ts#L102-L138","documentation":"The prompt-alignment scorer supports evaluation modes 'user', 'system', and 'both'. In 'user' mode the judge grades the assistant response against the user's prompt; if the run input contains no user message, the scorer throws at run time because there is no user prompt to measure alignment against.","triggerScenarios":"Running createPromptAlignmentScorerLLM with options.evaluationMode = 'user' (the default path) on a run whose input has no user-role message — getUserMessageFromRunInput returns undefined/empty.","commonSituations":"Scoring runs whose input is only a system prompt (e.g. classification setups that put everything in system); message arrays using non-standard roles ('human' instead of 'user') that the extractor doesn't recognize; passing pre-formatted prompt strings instead of message arrays.","solutions":["Ensure run.input includes at least one user-role message","If your prompts live in the system message, set evaluationMode to 'system' or 'both'","Normalize roles before scoring (map 'human' → 'user') if using non-standard message shapes"],"exampleFix":"// before\nawait scorer.run({ input: { messages: [{ role: 'system', content: 'Do X' }] }, output });\n// after\nawait scorer.run({ input: { messages: [{ role: 'system', content: 'Do X' }, { role: 'user', content: 'the actual request' }] }, output });","handlingStrategy":"validation","validationCode":"if (evaluationMode === 'user' && !getUserMessageFromRunInput(run.input)) {\n  throw new Error('Run has no user prompt; cannot score user-prompt alignment');\n}","typeGuard":"function hasUserPrompt(input) {\n  return Boolean(getUserMessageFromRunInput(input));\n}","tryCatchPattern":"try {\n  const result = await scorer.run({ input, output });\n} catch (e) {\n  if (e.message.includes('User prompt is required')) {\n    return { skipped: true, reason: 'run has no user prompt' };\n  }\n  throw e;\n}","preventionTips":["Match evaluationMode to the actual message composition of your runs","Normalize message roles before scoring ('human' → 'user')","Validate runs against expected message shape before eval batches"],"tags":["evals","runtime","validation","message-extraction"],"backgroundTag":"missing-required-scorer-option","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}