n8n-io/n8n · error · Error

Eval "${this.evalName}" requires either .check() or .judge()

Error message

Eval "${this.evalName}" requires either .check() or .judge()

What it means

Thrown by Eval.build() (invoked lazily on first .run()) when neither .check() nor .judge() was configured. An Eval must declare exactly one scoring strategy before it can execute.

Source

Thrown at packages/@n8n/agents/src/sdk/eval.ts:115

	 * Mutually exclusive with `.check()`.
	 */
	judge(fn: JudgeHandlerFn): this {
		if (this.checkFn) {
			throw new Error(`Eval "${this.evalName}": cannot use both .check() and .judge()`);
		}
		this.judgeFn = fn;
		return this;
	}

	/** The eval name. */
	get name(): string {
		return this.evalName;
	}

	/** @internal Build the eval into a runnable form. */
	protected build(): BuiltEval {
		if (!this.checkFn && !this.judgeFn) {
			throw new Error(`Eval "${this.evalName}" requires either .check() or .judge()`);
		}

		if (this.judgeFn && !this.modelId) {
			throw new Error(`Eval "${this.evalName}" uses .judge() but no .model() was set`);
		}

		const name = this.evalName;
		const desc = this.desc;

		if (this.checkFn) {
			const checkFn = this.checkFn;
			return {
				name,
				description: desc,
				evalType: 'check' as const,
				modelId: this.modelId ?? null,
				credentialName: this.credentialName ?? null,
				_run: async (input: EvalInput) => await checkFn(input),

View on GitHub (pinned to 5ac6606e81)

Solutions

1. Add .check(fn) for deterministic scoring or .judge(fn) plus .model(id) plus .credential(name) for LLM-as-judge.
2. Gate eval construction on the condition that selects a scoring mode so the builder always ends with one.
3. Assert in tests that new Eval(name) chains always terminate with .check or .judge.

Example fix

// before
const e = new Eval('json').description('validates json');
await e.run({ input: '', output: '{}', expected: '' });
// after
const e = new Eval('json').description('validates json').check(({ output }) => {
  try { JSON.parse(output); return { score: 1, reasoning: 'ok' }; }
  catch { return { score: 0, reasoning: 'bad' }; }
});

function assertEvalReady(e: { checkFn?: unknown; judgeFn?: unknown }) {
  if (!e.checkFn && !e.judgeFn) {
    throw new Error('Eval has no scoring function; call .check() or .judge()');
  }
}

function evalHasScorer(e: { checkFn?: unknown; judgeFn?: unknown }): boolean {
  return typeof e.checkFn === 'function' || typeof e.judgeFn === 'function';
}

When it happens

Trigger: Constructing an Eval, optionally setting .description/.model/.credential, then calling .run(input) or passing it to a runner that triggers build — without ever calling .check() or .judge().

Common situations: Builder chain interrupted by an exception or early return; conditionally setting the scoring function behind a flag that resolved false; copy-paste that dropped the scoring line.

Related errors

• Eval "${this.evalName}": cannot use both .check() and .judge
• Eval "${this.evalName}" uses .judge() but no .model() was se
• Guardrail "${this.name}" requires a type
• Guardrail "${this.name}" requires a strategy
• Tool name is required

AI-assisted analysis of n8n-io/n8n@5ac6606e81 (2026-08-12). Data as JSON: /api/errors/6b35a47a4fd8b8e0. Report an issue: GitHub.