{"record":{"id":"8ceff11cfc0031f9","repo":"n8n-io/n8n","slug":"langsmith-api-key-is-not-set-the-aia-arm-needs-l","errorCode":null,"errorMessage":"LANGSMITH_API_KEY is not set — the AIA arm needs LangSmith access to sum builder cost.","messagePattern":"LANGSMITH_API_KEY is not set — the AIA arm needs LangSmith access to sum builder cost\\.","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/@n8n/instance-ai/evaluations/cli/build-cost-report.ts","lineNumber":106,"sourceCode":"\n/** LangSmith request with 429/5xx backoff — a full-run join fires ~100 queries,\n *  which trips the API's rate limit without pacing. Honors Retry-After. */\nasync function lsFetch(url: string, init: RequestInit): Promise<Response> {\n\tlet delayMs = 2_000;\n\tfor (let attempt = 1; ; attempt++) {\n\t\tconst res = await fetch(url, init);\n\t\tif (res.ok || attempt >= 6 || (res.status !== 429 && res.status < 500)) return res;\n\t\tconst retryAfter = Number(res.headers.get('retry-after'));\n\t\tconst waitMs = Number.isFinite(retryAfter) && retryAfter > 0 ? retryAfter * 1_000 : delayMs;\n\t\tawait new Promise((resolve) => setTimeout(resolve, waitMs));\n\t\tdelayMs = Math.min(delayMs * 2, 30_000);\n\t}\n}\n\nasync function langsmithConfig(): Promise<LangSmithConfig> {\n\tconst { apiUrl, apiKey } = configFor();\n\tif (!apiKey) {\n\t\tthrow new Error(\n\t\t\t'LANGSMITH_API_KEY is not set — the AIA arm needs LangSmith access to sum builder cost.',\n\t\t);\n\t}\n\tconst workspaceId = await resolveEvalWorkspaceId();\n\treturn {\n\t\tapiUrl,\n\t\theaders: {\n\t\t\t'x-api-key': apiKey,\n\t\t\t'Content-Type': 'application/json',\n\t\t\t...(workspaceId ? { 'X-Tenant-Id': workspaceId } : {}),\n\t\t},\n\t};\n}\n\nasync function resolveTraceProjectId(ls: LangSmithConfig, projectName: string): Promise<string> {\n\tconst res = await lsFetch(\n\t\t`${ls.apiUrl}/api/v1/sessions?name=${encodeURIComponent(projectName)}`,\n\t\t{","sourceCodeStart":88,"sourceCodeEnd":124,"githubUrl":"https://github.com/n8n-io/n8n/blob/5ac6606e81f67bb9534255570cd4e86fd8101eee/packages/@n8n/instance-ai/evaluations/cli/build-cost-report.ts#L88-L124","documentation":"Thrown by langsmithConfig() in build-cost-report.ts:106 when configFor() returns no apiKey. The Instance AI Assistant (AIA) arm sums builder cost by joining each test case's build threads to root runs in the LangSmith trace project; without an API key, that thread join cannot run. The key is read via configFor() which sources LANGSMITH_ENDPOINT / LANGSMITH_API_KEY. Note this is resolved lazily — only arms that need the thread join (those with threadIds but no persisted buildCostUsdPerRun) trigger it; a persisted-spend-only comparison runs without credentials.","triggerScenarios":"Running build-cost-report.ts with at least one --results file whose test cases carry threadIds (AIA-origin) but no buildCostUsdPerRun field, and LANGSMITH_API_KEY is not set in the environment. Also triggered by --probe-thread <id> (which always needs LangSmith).","commonSituations":"Comparing an AIA eval run against an MCP run without sourcing .env.local, or in CI where the LangSmith secret is missing or scoped to a different job. The lazy resolution means the error surfaces only when a thread-join arm is encountered, which can be confusing.","solutions":["Export LANGSMITH_API_KEY (and LANGSMITH_ENDPOINT if non-default) before running the report: `export LANGSMITH_API_KEY=lsv2_pt_...`.","Load from dotenv: `dotenvx run -f .env.local -- pnpm tsx evaluations/cli/build-cost-report.ts --results ...`.","If you only want persisted-spend arms (MCP runs with buildCostUsdPerRun), ensure all input files have that field and no threadIds — then the key is never needed.","For --probe-thread mode, the key is always required; set it unconditionally."],"exampleFix":"// before\npnpm tsx evaluations/cli/build-cost-report.ts --results aia-run/eval-results.json\n// after\nexport LANGSMITH_API_KEY=lsv2_pt_...\npnpm tsx evaluations/cli/build-cost-report.ts --results aia-run/eval-results.json","handlingStrategy":"validation","validationCode":"// Before running build-cost-report for an AIA (thread-join) arm:\nif (!process.env.LANGSMITH_API_KEY) {\n  throw new Error('LANGSMITH_API_KEY required for thread-join (AIA) cost arms.');\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always source .env.local when running build-cost-report: dotenvx run -f .env.local -- ....","For AIA arms, set LANGSMITH_API_KEY unconditionally even if you think only MCP arms are present.","Use --probe-thread as a connectivity smoke test before a full report run."],"tags":["cli","environment","langsmith","instance-ai","secrets","cost-report"],"backgroundTag":null,"analyzedSha":"5ac6606e81f67bb9534255570cd4e86fd8101eee","analyzedAt":"2026-08-12T05:26:35.080Z","schemaVersion":2},"datasetVersion":"2026-08-13T14:17:21.547Z"}