mastra-ai/mastra · error · Error

createMultiTurnJudgeScorer: options.scale must be a finite…

Error message

createMultiTurnJudgeScorer: options.scale must be a finite number

What it means

createMultiTurnJudgeScorer defaults options.scale to 1 but rejects any value that is not a finite number (NaN, Infinity, -Infinity, or a non-numeric value coerced into the check). The scale defines the numeric range the LLM judge grades against, so an invalid scale would produce meaningless scores.

Solutions

1. Pass an explicit finite numeric scale, e.g. { scale: 5 }
2. Validate config values with Number.isFinite(scale) before constructing the scorer
3. Coerce string inputs with Number() and reject NaN before use
4. Omit scale entirely to use the default of 1

Example fix

// before
const scale = parseFloat(process.env.JUDGE_SCALE); // NaN if unset
const scorer = createMultiTurnJudgeScorer({ criterion, options: { scale } });
// after
const scale = process.env.JUDGE_SCALE ? Number(process.env.JUDGE_SCALE) : undefined;
if (scale !== undefined && !Number.isFinite(scale)) throw new Error('JUDGE_SCALE must be a finite number');
const scorer = createMultiTurnJudgeScorer({ criterion, options: scale !== undefined ? { scale } : undefined });

const scale = options?.scale ?? 1;
if (!Number.isFinite(scale)) throw new Error(`Invalid judge scale: ${scale}`);

function isValidScale(s) {
  return typeof s === 'number' && Number.isFinite(s) && s > 0;
}

try {
  const scorer = createMultiTurnJudgeScorer({ criterion, options });
} catch (e) {
  if (e.message.includes('scale must be a finite number')) {
    throw new Error(`Bad config: scale=${options?.scale} is not finite`, { cause: e });
  }
  throw e;
}

When it happens

Trigger: Calling createMultiTurnJudgeScorer({ criterion, options: { scale: Number.NaN } }) or { scale: Infinity }; commonly a value read from a config file or parsed string (e.g. parseFloat('10/5')) yields NaN/Infinity.

Common situations: Loading scale from an env var or JSON config without validating the parse; dividing by zero upstream producing Infinity; passing a string '5' from CLI args instead of the number 5.

Related errors

• Both baselineResponse and noisyQuery are required for Noise…
• Context array cannot be empty if provided
• Either context or contextExtractor is required for Context…
• NO_SCORERS_PROVIDED
• A custom capture schema requires an onExtracted hook that…

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/1643bdcd7df30130. Report an issue: GitHub.

Appendix: source

Thrown at packages/evals/src/scorers/llm/multi-turn-judge/index.ts:89

 * ```
 *
 * To persist scores, register an instance under the same id on the Mastra instance. Only the id is
 * used to resolve scorer metadata, so the registered instance's `criterion` can be a placeholder.
 */
export function createMultiTurnJudgeScorer({
  model,
  criterion,
  options,
}: {
  model: MastraModelConfig;
  /** What the conversation must satisfy, in plain English. */
  criterion: string;
  options?: MultiTurnJudgeScorerOptions;
}) {
  const scale = options?.scale ?? 1;

  if (!Number.isFinite(scale)) {
    throw new Error('createMultiTurnJudgeScorer: options.scale must be a finite number');
  }

  return createScorer<ScorerRunInputForLLMJudge, ScorerRunOutputForLLMJudge>({
    id: 'multi-turn-judge-scorer',
    name: 'Multi-turn Judge (LLM)',
    description: 'Grades every assistant turn of a conversation against a plain-English criterion',
    judge: {
      model,
      instructions: MULTI_TURN_JUDGE_INSTRUCTIONS,
    },
  })
    .analyze({
      description: 'Judge the whole conversation against the criterion',
      outputSchema: analyzeOutputSchema,
      createPrompt: ({ run }) => createAnalyzePrompt({ criterion, turns: getAssistantTurns(run.output) }),
    })
    .generateScore(({ results }) => {
      const analysis = results.analyzeStepResult as MultiTurnJudgeAnalysisResult | undefined;

View on GitHub (pinned to 75dd419e61)