mastra-ai/mastra · error · Error

A user or system prompt is required for combined alignment s

Error message

A user or system prompt is required for combined alignment scoring

What it means

In 'both' evaluation mode the prompt-alignment scorer judges the response against the user prompt and system prompt together. It requires at least one of the two; if neither is present in the run input, the scorer throws because there is nothing to align against.

Source

Thrown at packages/evals/src/scorers/llm/prompt-alignment/index.ts:126

    },
  })
    .analyze({
      description: 'Analyze prompt-response alignment across multiple dimensions',
      outputSchema: analyzeOutputSchema,
      createPrompt: ({ run }) => {
        const userPrompt = getUserMessageFromRunInput(run.input) ?? '';
        const systemPrompt = getCombinedSystemPrompt(run.input) ?? '';
        const agentResponse = getAssistantMessageFromRunOutput(run.output) ?? '';

        // Validation based on evaluation mode
        if (evaluationMode === 'user' && !userPrompt) {
          throw new Error('User prompt is required for user prompt alignment scoring');
        }
        if (evaluationMode === 'system' && !systemPrompt) {
          throw new Error('System prompt is required for system prompt alignment scoring');
        }
        if (evaluationMode === 'both' && !userPrompt && !systemPrompt) {
          throw new Error('A user or system prompt is required for combined alignment scoring');
        }
        if (!agentResponse) {
          throw new Error('Agent response is required for prompt alignment scoring');
        }

        return createAnalyzePrompt({
          userPrompt,
          systemPrompt,
          agentResponse,
          evaluationMode,
          conversationHistory: historyOptions && getConversationHistoryFromRunInput(run.input, historyOptions),
        });
      },
    })
    .generateScore(({ results }) => {
      const analysis = results.analyzeStepResult;

      if (!analysis) {

View on GitHub (pinned to 75dd419e61)

Solutions

1. Ensure run.input contains a user or system message before scoring
2. Score complete conversation runs rather than tool-only intermediate steps
3. Validate run input shape (message roles) before invoking the scorer

Example fix

// before
await scorer.run({ input: { messages: [{ role: 'tool', content: 'result...' }] }, output });
// after
if (!hasUserOrSystemPrompt(runInput)) throw new Error('Run input must include a user or system prompt');
await scorer.run({ input: fullConversationInput, output });

if (evaluationMode === 'both' && !getUserMessageFromRunInput(run.input) && !getCombinedSystemPrompt(run.input)) {
  throw new Error('Run has neither user nor system prompt; cannot score combined alignment');
}

function hasAnyPrompt(input) {
  return Boolean(getUserMessageFromRunInput(input) || getCombinedSystemPrompt(input));
}

try {
  const result = await scorer.run({ input, output });
} catch (e) {
  if (e.message.includes('combined alignment scoring')) {
    return { skipped: true, reason: 'no user or system prompt in run input' };
  }
  throw e;
}

When it happens

Trigger: Running the scorer with evaluationMode = 'both' where run.input contains neither a user message nor a system message — e.g. an input of only tool/tool-result messages or an empty message array.

Common situations: Scoring intermediate agent steps that contain only tool calls/results; passing an empty or malformed messages array; misconfigured run capture that dropped prompt messages from the recorded input.

Related errors

• User prompt is required for user prompt alignment scoring
• System prompt is required for system prompt alignment scorin
• Both original query and noisy response are required for eval
• RUN_EXPERIMENT_FAILED_NO_DATA_PROVIDED
• INVALID_DATA_ITEM

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/bba9a992f49563af. Report an issue: GitHub.