n8n-io/n8n · warning · Error

No scenarios match --tier "${args.tier}"

Error message

No scenarios match --tier "${args.tier}"

What it means

After loading test cases and applying the --tier filter, if no cases remain, the CLI throws rather than proceeding with an empty build list. The tier filter (filterSlugsByTier, line 431) keeps only cases whose `datasets` array includes the specified tier value, mirroring the eval --tier semantics. This fires when the tier value does not match any case's datasets.

Source

Thrown at packages/@n8n/instance-ai/evaluations/cli/build-mcp-manifest.ts:517

			const file = join(workflowDir, `${slug}.json`);
			if (!existsSync(file)) {
				console.log(`  [${slug}] skip: scenario file missing`);
				continue;
			}
			casesBySlug.set(slug, testCaseSchema.parse(readJson(file, `test case ${slug}`)).conversation);
		}
	}
	// Drop cases with no user turn to build from (e.g. `replay`-seeded only).
	for (const [slug, conv] of [...casesBySlug]) {
		if (!conv.some((t) => t.role === 'user' && t.text.trim().length > 0)) {
			console.log(`  [${slug}] skip: no user turn to build from`);
			casesBySlug.delete(slug);
		}
	}
	args.slugs = [...casesBySlug.keys()];
	if (args.slugs.length === 0) {
		throw new Error(
			args.tier ? `No scenarios match --tier "${args.tier}"` : 'No scenarios to build',
		);
	}

	const projectScopes = uniqueProjectScopes([
		args.buildCwd ? resolve(args.buildCwd) : undefined,
		repoRoot,
		repoRoot ? undefined : process.cwd(),
	]);
	// Removed on process exit by the staging module itself.
	const mcpConfigPath = stageMcpConfigFromClaudeJson(args.mcpServerName, projectScopes);

	const tasks: Array<{ slug: string; iteration: number }> = [];
	for (const slug of args.slugs) {
		for (let i = 1; i <= args.iterations; i++) {
			tasks.push({ slug, iteration: i });
		}
	}

View on GitHub (pinned to 5ac6606e81)

Solutions

1. Check the tier spelling: grep '"datasets"' data/workflows/*.json to see declared tiers
2. Remove --tier to build all cases regardless of dataset membership
3. List all distinct tier values: jq -r '.datasets[]?' data/workflows/*.json | sort -u

Example fix

# before
pnpm eval:build-mcp-manifest --tier mc

# after
pnpm eval:build-mcp-manifest --tier mcp

// Check tier availability before running the full build:
import { readdirSync, readFileSync } from 'fs';
const available = new Set<string>();
for (const f of readdirSync(workflowDir).filter(f => f.endsWith('.json'))) {
  const datasets = JSON.parse(readFileSync(join(workflowDir, f), 'utf-8')).datasets ?? [];
  for (const d of datasets) available.add(d);
}
if (!available.has(args.tier)) {
  throw new Error(`No scenarios match --tier "${args.tier}". Available: ${[...available].join(', ')}`);
}

When it happens

Trigger: Running `pnpm eval:build-mcp-manifest --tier mcp` when no case file in the workflow directory has 'mcp' in its datasets array. Also fires after langtracer source filtering if the suite's cases lack the tier.

Common situations: Typo in the tier name (e.g. 'mc' instead of 'mcp'). The tier exists conceptually but no cases in the directory declare it. The tier concept was renamed and old cases carry the old label.

Related errors

• No scenarios to build
• --max-attempts must be >= 1
• --source langtracer requires --suite <slug>
• Missing value for ${flag}
• Invalid integer for ${flag}

AI-assisted analysis of n8n-io/n8n@5ac6606e81 (2026-08-12). Data as JSON: /api/errors/3cf02027d839ebea. Report an issue: GitHub.