mastra-ai/mastra · error · Error

User prompt is required for user prompt alignment scoring

Error message

User prompt is required for user prompt alignment scoring

What it means

The prompt-alignment scorer supports evaluation modes 'user', 'system', and 'both'. In 'user' mode the judge grades the assistant response against the user's prompt; if the run input contains no user message, the scorer throws at run time because there is no user prompt to measure alignment against.

Solutions

  1. Ensure run.input includes at least one user-role message
  2. If your prompts live in the system message, set evaluationMode to 'system' or 'both'
  3. Normalize roles before scoring (map 'human' → 'user') if using non-standard message shapes

Example fix

// before
await scorer.run({ input: { messages: [{ role: 'system', content: 'Do X' }] }, output });
// after
await scorer.run({ input: { messages: [{ role: 'system', content: 'Do X' }, { role: 'user', content: 'the actual request' }] }, output });
Defensive patterns

Strategy: validation

Validate before calling

if (evaluationMode === 'user' && !getUserMessageFromRunInput(run.input)) {
  throw new Error('Run has no user prompt; cannot score user-prompt alignment');
}

Type guard

function hasUserPrompt(input) {
  return Boolean(getUserMessageFromRunInput(input));
}

Try / catch

try {
  const result = await scorer.run({ input, output });
} catch (e) {
  if (e.message.includes('User prompt is required')) {
    return { skipped: true, reason: 'run has no user prompt' };
  }
  throw e;
}

Prevention

When it happens

Trigger: Running createPromptAlignmentScorerLLM with options.evaluationMode = 'user' (the default path) on a run whose input has no user-role message — getUserMessageFromRunInput returns undefined/empty.

Common situations: Scoring runs whose input is only a system prompt (e.g. classification setups that put everything in system); message arrays using non-standard roles ('human' instead of 'user') that the extractor doesn't recognize; passing pre-formatted prompt strings instead of message arrays.

Related errors

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/dda31abc4965cd72. Report an issue: GitHub.

Appendix: source

Thrown at packages/evals/src/scorers/llm/prompt-alignment/index.ts:120

    id: 'prompt-alignment-scorer',
    name: 'Prompt Alignment (LLM)',
    description: 'Evaluates how well the agent response aligns with the intent and requirements of the user prompt',
    judge: {
      model,
      instructions: PROMPT_ALIGNMENT_INSTRUCTIONS,
    },
  })
    .analyze({
      description: 'Analyze prompt-response alignment across multiple dimensions',
      outputSchema: analyzeOutputSchema,
      createPrompt: ({ run }) => {
        const userPrompt = getUserMessageFromRunInput(run.input) ?? '';
        const systemPrompt = getCombinedSystemPrompt(run.input) ?? '';
        const agentResponse = getAssistantMessageFromRunOutput(run.output) ?? '';

        // Validation based on evaluation mode
        if (evaluationMode === 'user' && !userPrompt) {
          throw new Error('User prompt is required for user prompt alignment scoring');
        }
        if (evaluationMode === 'system' && !systemPrompt) {
          throw new Error('System prompt is required for system prompt alignment scoring');
        }
        if (evaluationMode === 'both' && !userPrompt && !systemPrompt) {
          throw new Error('A user or system prompt is required for combined alignment scoring');
        }
        if (!agentResponse) {
          throw new Error('Agent response is required for prompt alignment scoring');
        }

        return createAnalyzePrompt({
          userPrompt,
          systemPrompt,
          agentResponse,
          evaluationMode,
          conversationHistory: historyOptions && getConversationHistoryFromRunInput(run.input, historyOptions),
        });

View on GitHub (pinned to 75dd419e61)