{"record":{"id":"81bc015f7f58e459","repo":"mastra-ai/mastra","slug":"system-prompt-is-required-for-system-prompt-alignm","errorCode":null,"errorMessage":"System prompt is required for system prompt alignment scoring","messagePattern":"System prompt is required for system prompt alignment scoring","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/evals/src/scorers/llm/prompt-alignment/index.ts","lineNumber":123,"sourceCode":"    judge: {\n      model,\n      instructions: PROMPT_ALIGNMENT_INSTRUCTIONS,\n    },\n  })\n    .analyze({\n      description: 'Analyze prompt-response alignment across multiple dimensions',\n      outputSchema: analyzeOutputSchema,\n      createPrompt: ({ run }) => {\n        const userPrompt = getUserMessageFromRunInput(run.input) ?? '';\n        const systemPrompt = getCombinedSystemPrompt(run.input) ?? '';\n        const agentResponse = getAssistantMessageFromRunOutput(run.output) ?? '';\n\n        // Validation based on evaluation mode\n        if (evaluationMode === 'user' && !userPrompt) {\n          throw new Error('User prompt is required for user prompt alignment scoring');\n        }\n        if (evaluationMode === 'system' && !systemPrompt) {\n          throw new Error('System prompt is required for system prompt alignment scoring');\n        }\n        if (evaluationMode === 'both' && !userPrompt && !systemPrompt) {\n          throw new Error('A user or system prompt is required for combined alignment scoring');\n        }\n        if (!agentResponse) {\n          throw new Error('Agent response is required for prompt alignment scoring');\n        }\n\n        return createAnalyzePrompt({\n          userPrompt,\n          systemPrompt,\n          agentResponse,\n          evaluationMode,\n          conversationHistory: historyOptions && getConversationHistoryFromRunInput(run.input, historyOptions),\n        });\n      },\n    })\n    .generateScore(({ results }) => {","sourceCodeStart":105,"sourceCodeEnd":141,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/evals/src/scorers/llm/prompt-alignment/index.ts#L105-L141","documentation":"In 'system' evaluation mode the prompt-alignment scorer grades the response against the combined system prompt. If getCombinedSystemPrompt(run.input) returns empty, there is no system prompt to judge against, so the scorer throws at run time.","triggerScenarios":"Running the scorer with evaluationMode = 'system' on a run whose input contains no system-role messages — e.g. a plain user/assistant exchange scored with the wrong mode.","commonSituations":"Selecting mode 'system' in config while the scored conversations have no system message; system instructions embedded in the first user message instead of a system role; misconfigured extractor that ignores the system role.","solutions":["Ensure run.input contains a system-role message when using evaluationMode 'system'","Switch evaluationMode to 'user' if your runs have no system prompt","Verify getCombinedSystemPrompt's expected message format matches your run input structure"],"exampleFix":"// before\nconst scorer = createPromptAlignmentScorerLLM({ model, options: { evaluationMode: 'system' } });\nawait scorer.run({ input: { messages: [{ role: 'user', content: 'hi' }] }, output });\n// after\nconst mode = hasSystemPrompt(runInput) ? 'system' : 'user';\nconst scorer = createPromptAlignmentScorerLLM({ model, options: { evaluationMode: mode } });","handlingStrategy":"validation","validationCode":"if (evaluationMode === 'system' && !getCombinedSystemPrompt(run.input)) {\n  throw new Error('Run has no system prompt; cannot score system-prompt alignment');\n}","typeGuard":"function hasSystemPrompt(input) {\n  return Boolean(getCombinedSystemPrompt(input));\n}","tryCatchPattern":"try {\n  const result = await scorer.run({ input, output });\n} catch (e) {\n  if (e.message.includes('System prompt is required')) {\n    return { skipped: true, reason: 'run has no system prompt' };\n  }\n  throw e;\n}","preventionTips":["Only select mode 'system' for conversations that actually carry a system message","Keep system instructions in a system-role message, not the first user message","Derive evaluationMode from the run's content instead of hardcoding it"],"tags":["evals","runtime","validation","message-extraction"],"backgroundTag":"missing-required-scorer-option","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}