{"record":{"id":"4037fdd936241535","repo":"mastra-ai/mastra","slug":"run-experiment-scorer-failed-to-score-result","errorCode":"RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_RESULT","errorMessage":"Failed to run experiment: Error running scorer ${scorer.id}","messagePattern":"Failed to run experiment: Error running scorer (.+?)","errorType":"error_code","errorClass":"MastraError","httpStatus":null,"severity":"error","filePath":"packages/core/src/evals/run/index.ts","lineNumber":1101,"sourceCode":"\n  if (Array.isArray(scorers)) {\n    for (const scorer of scorers) {\n      try {\n        const score = await scorer.run({\n          input: targetResult.scoringData?.input,\n          output: targetResult.scoringData?.output,\n          groundTruth: item.groundTruth,\n          requestContext: item.requestContext,\n          scoreSource: 'experiment',\n          targetScope: 'span',\n          targetEntityType,\n          targetTraceId,\n          targetSpanId: targetResult.spanId,\n        });\n\n        scorerResults[scorer.id] = score;\n      } catch (error) {\n        throw new MastraError(\n          {\n            domain: 'SCORER',\n            id: 'RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_RESULT',\n            category: 'USER',\n            text: `Failed to run experiment: Error running scorer ${scorer.id}`,\n            details: {\n              scorerId: scorer.id,\n              item: JSON.stringify(item),\n            },\n          },\n          error,\n        );\n      }\n    }\n  } else if (isAgentScorerConfig(scorers)) {\n    // Handle agent scorer config (agent-level + trajectory scorers)\n    if (scorers.agent) {\n      const agentScorerResults: Record<string, any> = {};","sourceCodeStart":1083,"sourceCodeEnd":1119,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/core/src/evals/run/index.ts#L1083-L1119","documentation":"Thrown when a regular (non-agent, non-trajectory) scorer throws while scoring the target's result during an experiment run. The scorer run (scorer.run with target trace/span ids) raised, and the runner wraps it in a MastraError naming the failing scorer id. Category USER, domain SCORER.","triggerScenarios":"In runEvals/experiment execution, iterating experiment.scorers and calling scorer.run(...) on the target result: the scorer's process function throws (LLM call failure for LLM-judge scorers, reference data missing, extract/output schema mismatch).","commonSituations":"LLM-judge scorer using a misconfigured/unavailable model or API key; scorer extract returning undefined so process fails; scorer expecting `reference` but the dataset item lacks it; custom scorer code with a runtime bug.","solutions":["Read the wrapped cause to see the scorer's internal error.","Run the scorer directly via mastra.getScorer(...).run(...) against a saved trace/span to reproduce.","Check the scorer's judge model config (API key, model id) if it is an LLM-based scorer.","Ensure required scorer inputs (e.g. reference) exist on every dataset item.","Add defensive checks in the scorer's process/extract for missing fields."],"exampleFix":"// before\nscorers: { correctness } // judge model key missing at runtime\n// after: validate env before the run\nif (!process.env.OPENAI_API_KEY) throw new Error('OPENAI_API_KEY required for correctness scorer');\nscorers: { correctness }","handlingStrategy":"try-catch","validationCode":"// smoke-test each scorer once before a full experiment\nawait scorer.run({ input: 'ping', output: 'pong' });","typeGuard":"function isScorerError(e) {\n  return e instanceof Error && typeof e.message === 'string' && e.message.includes('Error running scorer');\n}","tryCatchPattern":"try {\n  await experiment.run();\n} catch (e) {\n  if (e?.id === 'RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_RESULT') {\n    console.error('Scorer failed, cause:', e.cause);\n    // optionally continue the run without this scorer\n  } else throw e;\n}","preventionTips":["Smoke-test LLM-judge scorers with one sample before large runs","Ensure judge model env keys are set for every scorer","Guard scorer extract/process against missing fields (reference, output)","Run scorers standalone via scorer.run on a saved result to verify"],"tags":["evals","scorer","error-wrapping"],"backgroundTag":"scorer-run-failed","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T08:17:16.595Z"}