{"record":{"id":"3cf02027d839ebea","repo":"n8n-io/n8n","slug":"no-scenarios-match-tier-args-tier","errorCode":null,"errorMessage":"No scenarios match --tier \"${args.tier}\"","messagePattern":"No scenarios match --tier \"(.+?)\"","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"warning","filePath":"packages/@n8n/instance-ai/evaluations/cli/build-mcp-manifest.ts","lineNumber":517,"sourceCode":"\t\t\tconst file = join(workflowDir, `${slug}.json`);\n\t\t\tif (!existsSync(file)) {\n\t\t\t\tconsole.log(`  [${slug}] skip: scenario file missing`);\n\t\t\t\tcontinue;\n\t\t\t}\n\t\t\tcasesBySlug.set(slug, testCaseSchema.parse(readJson(file, `test case ${slug}`)).conversation);\n\t\t}\n\t}\n\t// Drop cases with no user turn to build from (e.g. `replay`-seeded only).\n\tfor (const [slug, conv] of [...casesBySlug]) {\n\t\tif (!conv.some((t) => t.role === 'user' && t.text.trim().length > 0)) {\n\t\t\tconsole.log(`  [${slug}] skip: no user turn to build from`);\n\t\t\tcasesBySlug.delete(slug);\n\t\t}\n\t}\n\targs.slugs = [...casesBySlug.keys()];\n\tif (args.slugs.length === 0) {\n\t\tthrow new Error(\n\t\t\targs.tier ? `No scenarios match --tier \"${args.tier}\"` : 'No scenarios to build',\n\t\t);\n\t}\n\n\tconst projectScopes = uniqueProjectScopes([\n\t\targs.buildCwd ? resolve(args.buildCwd) : undefined,\n\t\trepoRoot,\n\t\trepoRoot ? undefined : process.cwd(),\n\t]);\n\t// Removed on process exit by the staging module itself.\n\tconst mcpConfigPath = stageMcpConfigFromClaudeJson(args.mcpServerName, projectScopes);\n\n\tconst tasks: Array<{ slug: string; iteration: number }> = [];\n\tfor (const slug of args.slugs) {\n\t\tfor (let i = 1; i <= args.iterations; i++) {\n\t\t\ttasks.push({ slug, iteration: i });\n\t\t}\n\t}\n","sourceCodeStart":499,"sourceCodeEnd":535,"githubUrl":"https://github.com/n8n-io/n8n/blob/5ac6606e81f67bb9534255570cd4e86fd8101eee/packages/@n8n/instance-ai/evaluations/cli/build-mcp-manifest.ts#L499-L535","documentation":"After loading test cases and applying the --tier filter, if no cases remain, the CLI throws rather than proceeding with an empty build list. The tier filter (filterSlugsByTier, line 431) keeps only cases whose `datasets` array includes the specified tier value, mirroring the eval --tier semantics. This fires when the tier value does not match any case's datasets.","triggerScenarios":"Running `pnpm eval:build-mcp-manifest --tier mcp` when no case file in the workflow directory has 'mcp' in its datasets array. Also fires after langtracer source filtering if the suite's cases lack the tier.","commonSituations":"Typo in the tier name (e.g. 'mc' instead of 'mcp'). The tier exists conceptually but no cases in the directory declare it. The tier concept was renamed and old cases carry the old label.","solutions":["Check the tier spelling: grep '\"datasets\"' data/workflows/*.json to see declared tiers","Remove --tier to build all cases regardless of dataset membership","List all distinct tier values: jq -r '.datasets[]?' data/workflows/*.json | sort -u"],"exampleFix":"# before\npnpm eval:build-mcp-manifest --tier mc\n\n# after\npnpm eval:build-mcp-manifest --tier mcp","handlingStrategy":"validation","validationCode":"// Check tier availability before running the full build:\nimport { readdirSync, readFileSync } from 'fs';\nconst available = new Set<string>();\nfor (const f of readdirSync(workflowDir).filter(f => f.endsWith('.json'))) {\n  const datasets = JSON.parse(readFileSync(join(workflowDir, f), 'utf-8')).datasets ?? [];\n  for (const d of datasets) available.add(d);\n}\nif (!available.has(args.tier)) {\n  throw new Error(`No scenarios match --tier \"${args.tier}\". Available: ${[...available].join(', ')}`);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["List available tiers before running: jq -r '.datasets[]?' data/workflows/*.json | sort -u","Double-check tier spelling against case files","Remove --tier if you want to build all cases"],"tags":["cli","validation","scenarios","tier","build-mcp-manifest"],"backgroundTag":null,"analyzedSha":"5ac6606e81f67bb9534255570cd4e86fd8101eee","analyzedAt":"2026-08-12T05:26:35.080Z","schemaVersion":2},"datasetVersion":"2026-08-12T13:17:24.610Z"}