mastra-ai/mastra · error · Error

Invalid scoring filter: path(s)

Error message

Invalid scoring filter: path(s) ${invalid.map(p => `"${p}"`).join(', ')} must start with one of: ${SCORING_PREDICATE_ROOTS.join(', ')}

What it means

Scoring filters (predicates on scorer bindings) may only reference paths rooted at requestContext, entity, entityType, source, threadId, resourceId, or projectId. `validateScoringPredicate` walks every path in the predicate and throws this Error at definition time (constructor / listScorers) if any path starts with a disallowed root — chiefly `input`/`output`, which are deliberately excluded because output-dependent filters cannot be evaluated against stored records at query time. This fails loud so filters don't silently skip all scoring at runtime.

Solutions

1. Rewrite each invalid path to start with one of: requestContext, entity, entityType, source, threadId, resourceId, projectId
2. Move input-dependent conditioning out of the filter (e.g. use sampling plus runtime scoring logic instead)
3. Fix typos to the exact scalar roots (e.g. `threadId` not `thread.id`)
4. For nested request data, use `requestContext.<key>` with dot-joined flat keys (e.g. `requestContext.user.tier`)

Example fix

// before
new ScorerBinding({
  scorer: toneScorer,
  filter: { path: 'output.score', gt: 0.5 }, // invalid root
});
// after
new ScorerBinding({
  scorer: toneScorer,
  filter: { path: 'requestContext.user.tier', eq: 'pro' },
});

const ROOTS = ['requestContext','entity','entityType','source','threadId','resourceId','projectId'];
export function checkScoringFilter(pred) {
  const paths = collectPaths(pred); // walk and/or/comparison nodes for .path
  const bad = paths.filter(p => !ROOTS.some(r => p === r || p.startsWith(r + '.')));
  if (bad.length) throw new Error(`Filter paths not allowed: ${bad.join(', ')}`);
}
checkScoringFilter(binding.filter); // before constructing the binding

try {
  const binding = new ScorerBinding({ scorer, filter });
} catch (e) {
  if (e.message.startsWith('Invalid scoring filter:')) {
    console.error('Rewrite filter paths to one of: requestContext, entity, entityType, source, threadId, resourceId, projectId');
  } else throw e;
}

When it happens

Trigger: Registering a scorer binding whose `filter` predicate references `input.x`, `output.y`, `run.foo`, or any typo'd root — thrown from the scorer-binding constructor or `mastra.listScorers()`.

Common situations: Writing a filter like `output.score > 0.5` hoping to conditionally score on results; typos such as `thread.id` instead of `threadId`; assuming request context fields are top-level instead of under `requestContext.`; upgrading from a version where output-rooted filters were tolerated.

Related errors

• maxQuestions must be at least 1 for Summarization scoring
• A custom capture schema requires an onExtracted hook that…
• AGENT_GENEREATE_SCORER_NOT_FOUND
• Agent response is required for prompt alignment scoring
• Both baselineResponse and noisyQuery are required for Noise…

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/f593f56f1ecfe2f9. Report an issue: GitHub.

Appendix: source

Thrown at packages/core/src/evals/predicate.ts:113

/**
 * Evaluate a scoring filter against a scoring context. Never throws for path
 * resolution failures — missing paths propagate to `false` on comparison ops
 * and `in` (fail closed: an unresolvable filter does not score), but to
 * `true` on the negated ops `notIn` and `notExists` (a missing value is
 * trivially "not in" any set), and to `false` on `exists`.
 */
export const evaluateScoringPredicate: (pred: Predicate, ctx: ScoringPredicateContext) => boolean =
  createPredicateEvaluator(resolveScoringPath);

/**
 * Validate a scoring filter at definition time. Throws if any referenced path
 * doesn't start with a known scoring root, so typos fail loud when the
 * binding is registered instead of silently skipping all scoring at runtime.
 */
export function validateScoringPredicate(pred: Predicate): void {
  const invalid = collectInvalidPredicatePaths(pred, SCORING_PREDICATE_ROOTS);
  if (invalid.length > 0) {
    throw new Error(
      `Invalid scoring filter: path(s) ${invalid.map(p => `"${p}"`).join(', ')} must start with one of: ${SCORING_PREDICATE_ROOTS.join(', ')}`,
    );
  }
}

View on GitHub (pinned to 75dd419e61)