mastra-ai/mastra · error · Error

No context available for evaluation

Error message

No context available for evaluation

What it means

This error fires at scoring time inside the context-precision scorer's pipeline. After resolving context — either from options.context or by invoking contextExtractor — if the resulting array is empty there is nothing for the LLM judge to rank, so the step throws. Unlike 2115/2116 (factory-time), this one occurs during run() when a per-run contextExtractor returns an empty array for that specific run.

Source

Thrown at packages/evals/src/scorers/llm/context-precision/index.ts:90

      'A scorer that evaluates the relevance and precision of retrieved context nodes for generating expected outputs',
    judge: {
      model,
      instructions: CONTEXT_PRECISION_AGENT_INSTRUCTIONS,
    },
    type: 'agent',
  })
    .analyze({
      description: 'Evaluate the relevance of each context piece for generating the expected output',
      outputSchema: contextRelevanceOutputSchema,
      createPrompt: ({ run }) => {
        const input = getUserMessageFromRunInput(run.input) ?? '';
        const output = getAssistantMessageFromRunOutput(run.output) ?? '';

        // Get context either from options or extractor
        const context = getContext({ input: run.input, output: run.output, options });

        if (context.length === 0) {
          throw new Error('No context available for evaluation');
        }

        return createContextRelevancePrompt({
          input,
          output,
          context,
        });
      },
    })
    .generateScore(({ results }) => {
      if (!results.analyzeStepResult || results.analyzeStepResult.verdicts.length === 0) {
        return 0;
      }

      const verdicts = results.analyzeStepResult.verdicts;

      // Sort verdicts by context_index to ensure proper order
      const sortedVerdicts = verdicts.sort((a, b) => a.context_index - b.context_index);

View on GitHub (pinned to 75dd419e61)

Solutions

1. Fix or harden the contextExtractor so it falls back to a valid context source when the primary field is absent
2. Pre-filter runs: skip scoring (or mark as errored) for runs whose extractor yields no context
3. Inspect the failing run's input to confirm where retrieved context should live and correct the extractor's field access
4. If the static context option is what you intended, ensure contextExtractor is not overriding it with an empty result

Example fix

// before
options: { contextExtractor: ({ run }) => run.input.retrievedDocuments.map(d => d.text) }

// after
options: {
  contextExtractor: ({ run }) => {
    const docs = run.input.retrievedDocuments ?? [];
    if (!docs.length) throw new Error('Run has no retrieved documents; skipping');
    return docs.map(d => d.text);
  },
}

const ctx = options.contextExtractor?.({ run }) ?? options.context ?? [];
if (!ctx.length) {
  console.warn('Skipping run: no context resolved for evaluation');
  return null;
}

try {
  const result = await scorer.run(payload);
  return result;
} catch (err) {
  if ((err as Error).message === 'No context available for evaluation') {
    markRunErrored(payload, 'no context resolved');
    return null;
  }
  throw err;
}

When it happens

Trigger: Calling score()/run() where options.contextExtractor({ run }) returns [] for that run (e.g. run.input has no retrievedDocuments); passing dynamic context that resolves to an empty array for a particular sample.

Common situations: RAG evaluation where some runs had no retrieval hits; contextExtractor reading a field with the wrong name/shape so it always maps to []; batch scoring with mixed datasets where some rows lack context.

Related errors

• Either context or contextExtractor is required for Context P
• Context array cannot be empty if provided
• MASTR_SCORER_FAILED_TO_CREATE_MISSING_ID
• MASTR_SCORER_FAILED_TO_RUN_MISSING_GENERATE_SCORE
• MASTR_SCORER_FAILED_TO_RUN_WORKFLOW_FAILED

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/732f8d5ce5da8c1f. Report an issue: GitHub.