mastra-ai/mastra · error · Error

Agent response is required for prompt alignment scoring

Error message

Agent response is required for prompt alignment scoring

What it means

The prompt-alignment LLM scorer requires an agentResponse to analyze, because the scorer compares the user/system prompt against what the agent actually produced. During scorer creation (createPromptAlignmentScorerLLM), if evaluationMode is 'user', 'system', or 'both' but agentResponse is missing, the factory throws immediately instead of silently producing a meaningless score. It is a fail-fast guard so misconfigured scorers never run.

Source

Thrown at packages/evals/src/scorers/llm/prompt-alignment/index.ts:129

      description: 'Analyze prompt-response alignment across multiple dimensions',
      outputSchema: analyzeOutputSchema,
      createPrompt: ({ run }) => {
        const userPrompt = getUserMessageFromRunInput(run.input) ?? '';
        const systemPrompt = getCombinedSystemPrompt(run.input) ?? '';
        const agentResponse = getAssistantMessageFromRunOutput(run.output) ?? '';

        // Validation based on evaluation mode
        if (evaluationMode === 'user' && !userPrompt) {
          throw new Error('User prompt is required for user prompt alignment scoring');
        }
        if (evaluationMode === 'system' && !systemPrompt) {
          throw new Error('System prompt is required for system prompt alignment scoring');
        }
        if (evaluationMode === 'both' && !userPrompt && !systemPrompt) {
          throw new Error('A user or system prompt is required for combined alignment scoring');
        }
        if (!agentResponse) {
          throw new Error('Agent response is required for prompt alignment scoring');
        }

        return createAnalyzePrompt({
          userPrompt,
          systemPrompt,
          agentResponse,
          evaluationMode,
          conversationHistory: historyOptions && getConversationHistoryFromRunInput(run.input, historyOptions),
        });
      },
    })
    .generateScore(({ results }) => {
      const analysis = results.analyzeStepResult;

      if (!analysis) {
        // Default to 0 if analysis failed
        return 0;
      }

View on GitHub (pinned to 75dd419e61)

Solutions

Pass the agent's response string when creating the scorer: agentResponse: result.text (or your stored response variable).
If the response comes from a run, extract it from the run output before calling the scorer and assert it is a non-empty string.
Check your variable mapping/renaming — a typo like 'reponse' silently yields undefined and trips this check.

Example fix

// before
const scorer = createPromptAlignmentScorerLLM({
  model: 'openai/gpt-4o',
  evaluationMode: 'both',
  userPrompt,
  systemPrompt,
});
// after
const scorer = createPromptAlignmentScorerLLM({
  model: 'openai/gpt-4o',
  evaluationMode: 'both',
  userPrompt,
  systemPrompt,
  agentResponse: await agent.generate(userPrompt).then((r) => r.text),
});

Defensive patterns

Strategy: validation

Validate before calling

function assertScorerInputs(opts: { agentResponse?: string | null }) {
  if (!opts.agentResponse || opts.agentResponse.trim().length === 0) {
    throw new Error('createPromptAlignmentScorerLLM requires a non-empty agentResponse');
  }
}

Type guard

const hasAgentResponse = (r: unknown): r is string => typeof r === 'string' && r.trim().length > 0;

Try / catch

try {
  const scorer = createPromptAlignmentScorerLLM({ ...opts, agentResponse });
} catch (err) {
  if (err instanceof Error && err.message.includes('Agent response is required')) {
    // log config problem and skip scorer creation
  } else throw err;
}

Prevention

Extract agentResponse from the run output immediately after generation and pass it in the same call site.
Add a unit test asserting the scorer factory receives a non-empty response string.
Use TypeScript non-optional typed inputs so omission is a compile error.

When it happens

Trigger: Calling createPromptAlignmentScorerLLM (or the scorer factory) with no agentResponse value, e.g. agentResponse omitted, undefined, or an empty string, after the prompt checks pass (evaluationMode 'user' has userPrompt, 'system' has systemPrompt, or 'both' has at least one prompt).

Common situations: Wiring the scorer into an eval where the run output field mapping is wrong so agentResponse ends up undefined; building the scorer before the agent has run; renaming a variable and forgetting to pass the response; constructing the scorer in a dry-run/test harness that only supplies prompts.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/2774a8c959dedcde. Report an issue: GitHub.