n8n-io/n8n · error · Error

Eval "${this.evalName}" uses .judge() but no .model() was se

Error message

Eval "${this.evalName}" uses .judge() but no .model() was set

What it means

First of two guards enforcing that LLM-as-judge evals declare a model. Thrown at the top of Eval.build() when this.judgeFn is set but this.modelId is undefined. The builder cannot construct the judge AgentRuntime without a model id.

Source

Thrown at packages/@n8n/agents/src/sdk/eval.ts:119

			throw new Error(`Eval "${this.evalName}": cannot use both .check() and .judge()`);
		}
		this.judgeFn = fn;
		return this;
	}

	/** The eval name. */
	get name(): string {
		return this.evalName;
	}

	/** @internal Build the eval into a runnable form. */
	protected build(): BuiltEval {
		if (!this.checkFn && !this.judgeFn) {
			throw new Error(`Eval "${this.evalName}" requires either .check() or .judge()`);
		}

		if (this.judgeFn && !this.modelId) {
			throw new Error(`Eval "${this.evalName}" uses .judge() but no .model() was set`);
		}

		const name = this.evalName;
		const desc = this.desc;

		if (this.checkFn) {
			const checkFn = this.checkFn;
			return {
				name,
				description: desc,
				evalType: 'check' as const,
				modelId: this.modelId ?? null,
				credentialName: this.credentialName ?? null,
				_run: async (input: EvalInput) => await checkFn(input),
			};
		}

		// LLM-as-judge mode

View on GitHub (pinned to 5ac6606e81)

Solutions

1. Call .model('anthropic/claude-haiku-4-5') (or another model id) before .judge().
2. Also call .credential(name) so the judge runtime can resolve an API key.
3. Validate eval config in tests by asserting modelId is set whenever judgeFn is.

Example fix

// before
const e = new Eval('corr').judge(async ({ llm }) => { ... });
// after
const e = new Eval('corr')
  .model('anthropic/claude-haiku-4-5')
  .credential('anthropic')
  .judge(async ({ llm }) => { ... });

function assertJudgeModel(e: { judgeFn?: unknown; modelId?: string }) {
  if (e.judgeFn && !e.modelId) {
    throw new Error('Eval uses .judge() without .model(); add .model(provider/name)');
  }
}

function judgeEvalHasModel(e: { judgeFn?: unknown; modelId?: string }): boolean {
  return typeof e.judgeFn === 'function' && typeof e.modelId === 'string' && e.modelId.length > 0;
}

When it happens

Trigger: Calling .judge(fn) without a preceding .model('provider/name'), then triggering build via .run() or a runner.

Common situations: Forgetting .model() when porting a check-style eval to judge; setting .model(undefined) conditionally; relying on a parent agent's model to propagate (it does not).

Related errors

• Eval "${this.evalName}": cannot use both .check() and .judge
• Eval "${this.evalName}" requires either .check() or .judge()
• Agent "${this.name}" requires a model
• Guardrail "${this.name}" requires a type
• Guardrail "${this.name}" requires a strategy

AI-assisted analysis of n8n-io/n8n@5ac6606e81 (2026-08-12). Data as JSON: /api/errors/a07039c30d726d08. Report an issue: GitHub.