{"record":{"id":"880369489f27e824","repo":"n8n-io/n8n","slug":"workflow-directory-not-found-workflowdir","errorCode":null,"errorMessage":"Workflow directory not found: ${workflowDir}","messagePattern":"Workflow directory not found: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/@n8n/instance-ai/evaluations/cli/build-mcp-manifest.ts","lineNumber":407,"sourceCode":"\t\t\ttotalCostUSD: sum((r) => r.cost),\n\t\t\tavgDurationMs: total > 0 ? sum((r) => r.durationMs) / total : 0,\n\t\t},\n\t\tbuilds: successful.map((r) => ({\n\t\t\tslug: r.slug,\n\t\t\titeration: r.iteration,\n\t\t\tworkflowId: r.workflowId,\n\t\t\tturns: r.turns,\n\t\t\tcostUSD: r.cost,\n\t\t\tdurationMs: r.durationMs,\n\t\t})),\n\t};\n\n\twriteFileSync(args.statsPath, JSON.stringify(stats, null, 2));\n}\n\nfunction discoverSlugs(workflowDir: string): string[] {\n\tif (!existsSync(workflowDir)) {\n\t\tthrow new Error(`Workflow directory not found: ${workflowDir}`);\n\t}\n\treturn readdirSync(workflowDir)\n\t\t.filter((f) => f.endsWith('.json'))\n\t\t.map((f) => basename(f, '.json'))\n\t\t.sort();\n}\n\nconst tierDatasetsSchema = z.object({ datasets: z.array(z.string()).optional() }).passthrough();\n\n/** A test case's `datasets`, defaulting to the shared eval default when absent — mirrors the loader schema. */\nfunction readDatasets(workflowDir: string, slug: string): string[] {\n\tconst file = join(workflowDir, `${slug}.json`);\n\tif (!existsSync(file)) return [];\n\ttry {\n\t\treturn (\n\t\t\ttierDatasetsSchema.parse(readJson(file, `test case ${slug}`)).datasets ?? DEFAULT_DATASETS\n\t\t);\n\t} catch {","sourceCodeStart":389,"sourceCodeEnd":425,"githubUrl":"https://github.com/n8n-io/n8n/blob/5ac6606e81f67bb9534255570cd4e86fd8101eee/packages/@n8n/instance-ai/evaluations/cli/build-mcp-manifest.ts#L389-L425","documentation":"The discoverSlugs function (line 405) lists all .json files in the workflow directory to build test cases when no explicit slugs are provided. If the directory does not exist, it throws rather than returning an empty list (which would silently produce 'No scenarios to build'). The directory defaults to packages/@n8n/instance-ai/evaluations/data/workflows/ under the repo root, or can be overridden with --workflow-dir.","triggerScenarios":"Running with --source disk (the default) when the default workflow directory does not exist (e.g. a shallow/partial checkout missing the data folder), or passing --workflow-dir <nonexistent-path>.","commonSituations":"Running from outside the n8n repo without --workflow-dir. A typo in the --workflow-dir path. The data/workflows directory has not been created yet or was cleaned by pnpm reset --full.","solutions":["Pass --workflow-dir with the correct absolute path to your test-case JSON directory","Run from inside the n8n repo so the default path resolves via git rev-parse","Verify the directory exists: ls <path>","Use --source langtracer to pull test cases from a remote suite instead of disk"],"exampleFix":"# before\npnpm eval:build-mcp-manifest --workflow-dir /wrong/path\n\n# after\npnpm eval:build-mcp-manifest --workflow-dir packages/@n8n/instance-ai/evaluations/data/workflows","handlingStrategy":"validation","validationCode":"import { existsSync } from 'fs';\nconst workflowDir = args.workflowDir ?? defaultDir;\nif (!existsSync(workflowDir)) {\n  throw new Error(`Workflow directory not found: ${workflowDir}`);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Verify the workflow directory exists with ls before running","When running outside the repo, always pass --workflow-dir explicitly","Use absolute paths to avoid ambiguity"],"tags":["cli","filesystem","path","validation","build-mcp-manifest"],"backgroundTag":null,"analyzedSha":"5ac6606e81f67bb9534255570cd4e86fd8101eee","analyzedAt":"2026-08-12T05:26:35.080Z","schemaVersion":2},"datasetVersion":"2026-08-12T13:17:24.610Z"}