n8n-io/n8n · error · Error

No test cases match --tier "${tier}". Known tiers: ${known.j

Error message

No test cases match --tier "${tier}". Known tiers: ${known.join(', ') || '(none)'}.

What it means

Thrown by `loadTestCases()` after the disk source loaded cases and the `--tier` filter produced zero matches. Each test case declares a `datasets` array naming the tiers it belongs to; the filter keeps only cases whose `datasets` includes the requested tier. The message lists all known tiers across the loaded set (deduped, sorted) so you can pick a valid one. Note this only runs for the disk source; LangTracer tier filtering happens upstream in the provider.

Source

Thrown at packages/@n8n/instance-ai/evaluations/data/source.ts:35

			suite: args.suite,
			filter: args.filter,
			exclude: args.exclude,
			tier: args.tier,
			logger,
		});
	}

	const cases = [
		...loadWorkflowTestCasesWithFiles(args.filter, args.exclude),
		...loadAgentEvalTestCasesWithFiles(args.filter, args.exclude),
	];
	const { tier } = args;
	if (!tier) return cases;

	const matched = cases.filter(({ testCase }) => testCase.datasets.includes(tier));
	if (matched.length === 0) {
		const known = [...new Set(cases.flatMap(({ testCase }) => testCase.datasets))].sort();
		throw new Error(
			`No test cases match --tier "${tier}". Known tiers: ${known.join(', ') || '(none)'}.`,
		);
	}
	return matched;
}

View on GitHub (pinned to 5ac6606e81)

Solutions

1. Re-run with one of the tiers listed in the error message (the `Known tiers: ...` portion).
2. If the tier should exist, open the case files you expect to include it and confirm their `datasets` arrays contain the tier; add it if missing.
3. If `--filter`/`--exclude` shrank the set, widen them or drop `--tier` to run all matched cases.
4. If you intended to run cases with NO tier filter, omit `--tier` entirely.

Example fix

// before
$ node eval.ts --tier smoek
// Error: ... Known tiers: nightly, smoke-tests, full
// after
$ node eval.ts --tier smoke-tests

function knownTiers(cases: WorkflowTestCaseWithFile[]): string[] {
  return [...new Set(cases.flatMap((c) => c.testCase.datasets))].sort();
}
// before filtering
if (tier && !knownTiers(cases).includes(tier)) {
  throw new Error(`Unknown tier '${tier}'. Known: ${knownTiers(cases).join(', ')}`);
}

When it happens

Trigger: Pass `--tier foo` when no case lists `foo` in its `datasets`. Typo a tier name (`--tier smoke` when the case declares `smoke-tests`). A case file's `datasets` field was renamed/emptied, removing a tier that a CI script still requests. The `--filter`/`--exclude` flags reduced the case set to one that happens to have no cases in the requested tier.

Common situations: CI matrix passes a tier name that drifted from the case files. A team renames a tier and forgets to update the CI config. Combining `--filter` and `--tier` such that the filtered subset has no members in that tier.

Related errors

• Unknown flag: ${arg.split('=', 1)[0]}
• Unexpected positional argument
• Missing value for ${flag}
• --source langtracer requires --suite <slug>
• Missing value for ${arg}

AI-assisted analysis of n8n-io/n8n@5ac6606e81 (2026-08-12). Data as JSON: /api/errors/5fcf776df4d94a46. Report an issue: GitHub.