{"record":{"id":"5fcf776df4d94a46","repo":"n8n-io/n8n","slug":"no-test-cases-match-tier-tier-known-tiers","errorCode":null,"errorMessage":"No test cases match --tier \"${tier}\". Known tiers: ${known.join(', ') || '(none)'}.","messagePattern":"No test cases match --tier \"(.+?)\"\\. Known tiers: (.+?)\\.","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/@n8n/instance-ai/evaluations/data/source.ts","lineNumber":35,"sourceCode":"\t\t\tsuite: args.suite,\n\t\t\tfilter: args.filter,\n\t\t\texclude: args.exclude,\n\t\t\ttier: args.tier,\n\t\t\tlogger,\n\t\t});\n\t}\n\n\tconst cases = [\n\t\t...loadWorkflowTestCasesWithFiles(args.filter, args.exclude),\n\t\t...loadAgentEvalTestCasesWithFiles(args.filter, args.exclude),\n\t];\n\tconst { tier } = args;\n\tif (!tier) return cases;\n\n\tconst matched = cases.filter(({ testCase }) => testCase.datasets.includes(tier));\n\tif (matched.length === 0) {\n\t\tconst known = [...new Set(cases.flatMap(({ testCase }) => testCase.datasets))].sort();\n\t\tthrow new Error(\n\t\t\t`No test cases match --tier \"${tier}\". Known tiers: ${known.join(', ') || '(none)'}.`,\n\t\t);\n\t}\n\treturn matched;\n}\n","sourceCodeStart":17,"sourceCodeEnd":41,"githubUrl":"https://github.com/n8n-io/n8n/blob/5ac6606e81f67bb9534255570cd4e86fd8101eee/packages/@n8n/instance-ai/evaluations/data/source.ts#L17-L41","documentation":"Thrown by `loadTestCases()` after the disk source loaded cases and the `--tier` filter produced zero matches. Each test case declares a `datasets` array naming the tiers it belongs to; the filter keeps only cases whose `datasets` includes the requested tier. The message lists all known tiers across the loaded set (deduped, sorted) so you can pick a valid one. Note this only runs for the disk source; LangTracer tier filtering happens upstream in the provider.","triggerScenarios":"Pass `--tier foo` when no case lists `foo` in its `datasets`. Typo a tier name (`--tier smoke` when the case declares `smoke-tests`). A case file's `datasets` field was renamed/emptied, removing a tier that a CI script still requests. The `--filter`/`--exclude` flags reduced the case set to one that happens to have no cases in the requested tier.","commonSituations":"CI matrix passes a tier name that drifted from the case files. A team renames a tier and forgets to update the CI config. Combining `--filter` and `--tier` such that the filtered subset has no members in that tier.","solutions":["Re-run with one of the tiers listed in the error message (the `Known tiers: ...` portion).","If the tier should exist, open the case files you expect to include it and confirm their `datasets` arrays contain the tier; add it if missing.","If `--filter`/`--exclude` shrank the set, widen them or drop `--tier` to run all matched cases.","If you intended to run cases with NO tier filter, omit `--tier` entirely."],"exampleFix":"// before\n$ node eval.ts --tier smoek\n// Error: ... Known tiers: nightly, smoke-tests, full\n// after\n$ node eval.ts --tier smoke-tests","handlingStrategy":"validation","validationCode":"function knownTiers(cases: WorkflowTestCaseWithFile[]): string[] {\n  return [...new Set(cases.flatMap((c) => c.testCase.datasets))].sort();\n}\n// before filtering\nif (tier && !knownTiers(cases).includes(tier)) {\n  throw new Error(`Unknown tier '${tier}'. Known: ${knownTiers(cases).join(', ')}`);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep tier names in a shared constant referenced by both case files and CI configs.","When renaming a tier, grep the repo and CI for the old name in the same change.","Print available tiers at the start of a CI job that uses `--tier` so failures are self-diagnosing."],"tags":["cli","argument-parsing","tier-filter","configuration","developer-error"],"backgroundTag":null,"analyzedSha":"5ac6606e81f67bb9534255570cd4e86fd8101eee","analyzedAt":"2026-08-12T05:26:35.080Z","schemaVersion":2},"datasetVersion":"2026-08-12T18:17:37.767Z"}