mastra-ai/mastra · error · Error

maxQuestions must be at least 1 for Summarization scoring

Error message

maxQuestions must be at least 1 for Summarization scoring

What it means

The summarization scorer generates questions from a summary to verify it is faithful; maxQuestions controls how many are generated. A value below 1 (0 or negative) makes question generation impossible, so createSummarizationScorer throws at construction time. Only an explicitly provided options.maxQuestions is validated; omitted values fall back to DEFAULT_MAX_QUESTIONS.

Source

Thrown at packages/evals/src/scorers/llm/summarization/index.ts:194

 * @example
 * ```ts
 * const scorer = createSummarizationScorer({
 *   model: 'openai/gpt-5.5',
 *   options: { maxQuestions: 10 },
 * });
 *
 * const result = await scorer.run(run);
 * ```
 */
export function createSummarizationScorer({
  model,
  options = {},
}: {
  model: MastraModelConfig;
  options?: SummarizationMetricOptions;
}) {
  if (options.maxQuestions !== undefined && options.maxQuestions < 1) {
    throw new Error('maxQuestions must be at least 1 for Summarization scoring');
  }

  const maxQuestions = options.maxQuestions ?? DEFAULT_MAX_QUESTIONS;

  return createScorer<ScorerRunInputForLLMJudge, ScorerRunOutputForLLMJudge>({
    id: 'summarization-scorer',
    name: 'Summarization Scorer',
    description:
      'A scorer that evaluates whether a summary stays faithful to its source text and preserves the information the source states',
    judge: {
      model,
      instructions: SUMMARIZATION_AGENT_INSTRUCTIONS,
    },
    type: 'agent',
  })
    .preprocess({
      description: 'Judge each summary claim against the source and draw coverage questions from it',
      outputSchema: sourceJudgementOutputSchema,

View on GitHub (pinned to 75dd419e61)

Solutions

Set options.maxQuestions to a positive integer, e.g. maxQuestions: 5.
Remove the maxQuestions option entirely to use the library default.
Clamp before passing: Math.max(1, configuredMaxQuestions || DEFAULT).

Example fix

// before
const scorer = createSummarizationScorer({ model, options: { maxQuestions: 0 } });
// after
const scorer = createSummarizationScorer({ model, options: { maxQuestions: 5 } });

Defensive patterns

Strategy: validation

Validate before calling

const DEFAULT_MAX_QUESTIONS = 5;
const maxQuestions = options.maxQuestions ?? DEFAULT_MAX_QUESTIONS;
if (!Number.isInteger(maxQuestions) || maxQuestions < 1) {
  throw new RangeError(`maxQuestions must be a positive integer, got ${maxQuestions}`);
}

Type guard

const isValidMaxQuestions = (n: unknown): n is number => typeof n === 'number' && Number.isInteger(n) && n >= 1;

Try / catch

try {
  const scorer = createSummarizationScorer({ model, options });
} catch (err) {
  if (err instanceof Error && err.message.includes('maxQuestions must be at least 1')) {
    options = { ...options, maxQuestions: 5 }; // retry with default
  } else throw err;
}

Prevention

Clamp user/config-provided values with Math.max(1, value) before constructing the scorer.
Never use 0 to mean 'default'; omit the option instead.
Validate env/config numbers at startup with a schema (e.g. zod .int().min(1)).

When it happens

Trigger: Calling createSummarizationScorer({ model, options: { maxQuestions: 0 } }) or with any negative number, e.g. maxQuestions: -1, or a maxQuestions computed from an expression that evaluates to <= 0.

Common situations: Loading maxQuestions from config/env where the default is 0; computing maxQuestions from a ratio of summary length that floors to 0; a user entering 0 in a UI thinking it means 'unlimited'.

Understand the failure class

Background: "must be positive", "Invalid value": how libraries reject invalid parameter values (ValueError, ArgumentError, INVALID_PARAMETER_VALUE) — this error's family across 28 libraries.

Related errors

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/411d28cc23ffc073. Report an issue: GitHub.