{"record":{"id":"931538b3c3d61dc8","repo":"n8n-io/n8n","slug":"pass-results-eval-results-json-repeatable-on","errorCode":null,"errorMessage":"Pass --results <eval-results.json> (repeatable, one per arm) or --probe-thread <id>.","messagePattern":"Pass --results <eval-results\\.json> \\(repeatable, one per arm\\) or --probe-thread <id>\\.","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/@n8n/instance-ai/evaluations/cli/build-cost-report.ts","lineNumber":448,"sourceCode":"\tconst probeThread = single(flags, 'probe-thread');\n\tif (probeThread) {\n\t\tconst ls = await langsmithConfig();\n\t\tconst projectId = await resolveTraceProjectId(ls, traceProject);\n\t\tconst cost = await sumThreadCost(ls, projectId, probeThread);\n\t\tif (!cost) {\n\t\t\tconsole.log(`thread ${probeThread}: no runs found in trace project \"${traceProject}\"`);\n\t\t\treturn;\n\t\t}\n\t\tconsole.log(\n\t\t\t`thread ${probeThread}: ${cost.turns} turns, ${cost.tokens.toLocaleString()} tokens, ${fmtUsd(cost.costUsd)}`,\n\t\t);\n\t\treturn;\n\t}\n\n\tconst resultPaths = flags.get('results') ?? [];\n\tconst labels = flags.get('label') ?? [];\n\tif (resultPaths.length === 0) {\n\t\tthrow new Error(\n\t\t\t'Pass --results <eval-results.json> (repeatable, one per arm) or --probe-thread <id>.',\n\t\t);\n\t}\n\n\t// LangSmith access is resolved once, lazily — only when some arm needs the\n\t// thread join (a persisted-spend-only comparison runs without credentials).\n\tlet ls: LangSmithConfig | undefined;\n\tlet projectId: string | undefined;\n\n\tconst arms: ArmSummary[] = [];\n\tfor (let i = 0; i < resultPaths.length; i++) {\n\t\tconst results = loadResults(resultPaths[i]);\n\t\tconst label = labels[i] ?? results.experimentName ?? `arm${String(i + 1)}`;\n\t\tconst hasPersistedSpend = results.testCases.some((tc) => tc.buildCostUsdPerRun !== undefined);\n\t\tconst hasThreads = results.testCases.some((tc) =>\n\t\t\t(tc.threadIds ?? []).some((id) => id !== null),\n\t\t);\n\t\tif (hasPersistedSpend) {","sourceCodeStart":430,"sourceCodeEnd":466,"githubUrl":"https://github.com/n8n-io/n8n/blob/5ac6606e81f67bb9534255570cd4e86fd8101eee/packages/@n8n/instance-ai/evaluations/cli/build-cost-report.ts#L430-L466","documentation":"Thrown by main() in build-cost-report.ts:448 when neither --results nor --probe-thread is provided. The tool has two modes: a multi-arm cost comparison (one or more --results files) or a single-thread connectivity spot-check (--probe-thread <id>). With neither, there is nothing to do, so the parser fails fast with usage guidance.","triggerScenarios":"Invoking build-cost-report.ts with no arguments, or with only non-input flags (e.g. only --trace-project or --out but no --results and no --probe-thread).","commonSituations":"A developer runs the script expecting a default behavior, or an argument was lost during command construction. Also occurs when --results paths are all behind a shell expansion that produced nothing.","solutions":["Pass at least one --results <eval-results.json> for a cost comparison report (repeatable, one per arm).","Or pass --probe-thread <thread-id> for a single-thread cost spot-check.","Verify any shell variables expanding into --results paths are non-empty: `echo \"${RESULTS:?unset}\"."],"exampleFix":"// before\npnpm tsx evaluations/cli/build-cost-report.ts\n// after\npnpm tsx evaluations/cli/build-cost-report.ts --results mcp-run/eval-results.json --results aia-run/eval-results.json --label mcp --label aia","handlingStrategy":"validation","validationCode":"function validateReportInputs(args: string[]): void {\n  const hasResults = args.includes('--results');\n  const hasProbe = args.includes('--probe-thread');\n  if (!hasResults && !hasProbe) {\n    throw new Error('Pass --results <file> or --probe-thread <id>');\n  }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always pass at least one --results or --probe-thread.","Verify shell expansions producing --results paths are non-empty.","Use --probe-thread as a minimal smoke-test invocation."],"tags":["cli","argument-validation","instance-ai","cost-report"],"backgroundTag":null,"analyzedSha":"5ac6606e81f67bb9534255570cd4e86fd8101eee","analyzedAt":"2026-08-12T05:26:35.080Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}