mastra-ai/mastra · error · Error

createMultiTurnJudgeScorer: options.scale must be a finite n

Error message

createMultiTurnJudgeScorer: options.scale must be a finite number

What it means

createMultiTurnJudgeScorer defaults options.scale to 1 but rejects any value that is not a finite number (NaN, Infinity, -Infinity, or a non-numeric value coerced into the check). The scale defines the numeric range the LLM judge grades against, so an invalid scale would produce meaningless scores.

Source

Thrown at packages/evals/src/scorers/llm/multi-turn-judge/index.ts:89

 * ```
 *
 * To persist scores, register an instance under the same id on the Mastra instance. Only the id is
 * used to resolve scorer metadata, so the registered instance's `criterion` can be a placeholder.
 */
export function createMultiTurnJudgeScorer({
  model,
  criterion,
  options,
}: {
  model: MastraModelConfig;
  /** What the conversation must satisfy, in plain English. */
  criterion: string;
  options?: MultiTurnJudgeScorerOptions;
}) {
  const scale = options?.scale ?? 1;

  if (!Number.isFinite(scale)) {
    throw new Error('createMultiTurnJudgeScorer: options.scale must be a finite number');
  }

  return createScorer<ScorerRunInputForLLMJudge, ScorerRunOutputForLLMJudge>({
    id: 'multi-turn-judge-scorer',
    name: 'Multi-turn Judge (LLM)',
    description: 'Grades every assistant turn of a conversation against a plain-English criterion',
    judge: {
      model,
      instructions: MULTI_TURN_JUDGE_INSTRUCTIONS,
    },
  })
    .analyze({
      description: 'Judge the whole conversation against the criterion',
      outputSchema: analyzeOutputSchema,
      createPrompt: ({ run }) => createAnalyzePrompt({ criterion, turns: getAssistantTurns(run.output) }),
    })
    .generateScore(({ results }) => {
      const analysis = results.analyzeStepResult as MultiTurnJudgeAnalysisResult | undefined;

View on GitHub (pinned to 75dd419e61)

Solutions

1. Pass an explicit finite numeric scale, e.g. { scale: 5 }
2. Validate config values with Number.isFinite(scale) before constructing the scorer
3. Coerce string inputs with Number() and reject NaN before use
4. Omit scale entirely to use the default of 1

Example fix

// before
const scale = parseFloat(process.env.JUDGE_SCALE); // NaN if unset
const scorer = createMultiTurnJudgeScorer({ criterion, options: { scale } });
// after
const scale = process.env.JUDGE_SCALE ? Number(process.env.JUDGE_SCALE) : undefined;
if (scale !== undefined && !Number.isFinite(scale)) throw new Error('JUDGE_SCALE must be a finite number');
const scorer = createMultiTurnJudgeScorer({ criterion, options: scale !== undefined ? { scale } : undefined });

const scale = options?.scale ?? 1;
if (!Number.isFinite(scale)) throw new Error(`Invalid judge scale: ${scale}`);

function isValidScale(s) {
  return typeof s === 'number' && Number.isFinite(s) && s > 0;
}

try {
  const scorer = createMultiTurnJudgeScorer({ criterion, options });
} catch (e) {
  if (e.message.includes('scale must be a finite number')) {
    throw new Error(`Bad config: scale=${options?.scale} is not finite`, { cause: e });
  }
  throw e;
}

When it happens

Trigger: Calling createMultiTurnJudgeScorer({ criterion, options: { scale: Number.NaN } }) or { scale: Infinity }; commonly a value read from a config file or parsed string (e.g. parseFloat('10/5')) yields NaN/Infinity.

Common situations: Loading scale from an env var or JSON config without validating the parse; dividing by zero upstream producing Infinity; passing a string '5' from CLI args instead of the number 5.

Related errors

• NO_SCORERS_PROVIDED
• Either context or contextExtractor is required for Context R
• Context array cannot be empty if provided
• Both baselineResponse and noisyQuery are required for Noise
• Google RBAC roleMapping is required.

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/1643bdcd7df30130. Report an issue: GitHub.