{"record":{"id":"5759f6fb851cd65c","repo":"n8n-io/n8n","slug":"langsmith-runs-query-failed-res-status-await","errorCode":null,"errorMessage":"LangSmith runs/query failed: ${res.status} ${await res.text()}","messagePattern":"LangSmith runs/query failed: (.+?) (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/@n8n/instance-ai/evaluations/cli/build-cost-report.ts","lineNumber":171,"sourceCode":" *  --trace-project, or the trace hasn't landed yet). */\nexport async function sumThreadCost(\n\tls: LangSmithConfig,\n\tprojectId: string,\n\tthreadId: string,\n): Promise<ThreadCost | null> {\n\tconst res = await lsFetch(`${ls.apiUrl}/api/v1/runs/query`, {\n\t\tmethod: 'POST',\n\t\theaders: ls.headers,\n\t\tbody: JSON.stringify({\n\t\t\tsession: [projectId],\n\t\t\tis_root: true,\n\t\t\tfilter: `eq(thread_id, \"${threadId}\")`,\n\t\t\tlimit: RUNS_QUERY_LIMIT,\n\t\t\tselect: ['id', 'total_cost', 'total_tokens'],\n\t\t}),\n\t});\n\tif (!res.ok) {\n\t\tthrow new Error(`LangSmith runs/query failed: ${res.status} ${await res.text()}`);\n\t}\n\tconst { runs } = lsRunsQuerySchema.parse(await res.json());\n\tif (runs.length === 0) return null;\n\tif (runs.length === RUNS_QUERY_LIMIT) {\n\t\tconsole.warn(\n\t\t\t`thread ${threadId}: hit the runs/query page limit (${RUNS_QUERY_LIMIT}) — cost may be undercounted`,\n\t\t);\n\t}\n\treturn {\n\t\tcostUsd: runs.reduce((sum, r) => sum + (r.total_cost ?? 0), 0),\n\t\ttokens: runs.reduce((sum, r) => sum + (r.total_tokens ?? 0), 0),\n\t\tturns: runs.length,\n\t};\n}\n\n// ---------------------------------------------------------------------------\n// Per-case aggregation\n// ---------------------------------------------------------------------------","sourceCodeStart":153,"sourceCodeEnd":189,"githubUrl":"https://github.com/n8n-io/n8n/blob/5ac6606e81f67bb9534255570cd4e86fd8101eee/packages/@n8n/instance-ai/evaluations/cli/build-cost-report.ts#L153-L189","documentation":"Thrown by sumThreadCost() (build-cost-report.ts:171) when the LangSmith /api/v1/runs/query POST returns a non-OK status after the lsFetch backoff. This query joins a specific thread_id to its root runs (one per build turn) within the resolved session/project to sum total_cost and total_tokens. Like the sessions lookup, it has 429/5xx backoff, so this error indicates a hard failure or exhausted retries. The response status and body are echoed for diagnosis.","triggerScenarios":"During a full-run thread join (threadJoinCosts), each thread fires a runs/query. A single thread's query hitting a hard 4xx (auth, permissions, malformed filter) or a 5xx that exhausted 6 backoff attempts trips this. A full-run join fires ~100 queries, increasing the chance one trips the rate limit and exhausts backoff.","commonSituations":"Rate-limiting (429) that exhausted the 6-attempt exponential backoff, a transient 5xx storm, an auth key that was valid for sessions but scoped/restricted for runs/query, or a malformed thread_id producing a 4xx. High-concurrency joins (--concurrency large) can amplify rate-limit pressure.","solutions":["Retry the command — transient 5xx/429 storms often clear on the next run.","Lower --concurrency to reduce rate-limit pressure on the runs/query endpoint (default 3).","For 401/403, regenerate LANGSMITH_API_KEY or check its scopes.","If a specific thread_id is malformed, inspect the eval-results.json threadIds array for corruption.","Check the LangSmith status page for ongoing incidents."],"exampleFix":"// before (high concurrency tripping rate limits)\npnpm tsx evaluations/cli/build-cost-report.ts --results run/eval-results.json --concurrency 16\n// after\npnpm tsx evaluations/cli/build-cost-report.ts --results run/eval-results.json --concurrency 3","handlingStrategy":"retry","validationCode":"// Lower concurrency to avoid rate limits; the built-in lsFetch already retries 429/5xx.\n// Pass --concurrency 1 or 2 when joining many threads.\nconst concurrency = Math.min(desiredConcurrency, 2);","typeGuard":null,"tryCatchPattern":"// Retry the whole report on transient runs/query failures (lsFetch already retries per-request):\nfor (let attempt = 1; attempt <= 3; attempt++) {\n  try {\n    await runReport(args);\n    break;\n  } catch (error) {\n    const msg = error instanceof Error ? error.message : String(error);\n    if (msg.includes('runs/query failed') && attempt < 3) {\n      await new Promise((r) => setTimeout(r, 10_000 * attempt));\n      continue;\n    }\n    throw error;\n  }\n}","preventionTips":["Use a low --concurrency (1-3) for large thread joins to stay under rate limits.","Retry the full report on transient 5xx — lsFetch's per-request backoff is bounded.","Monitor LangSmith status page during large joins.","Ensure LANGSMITH_API_KEY has runs/query scope."],"tags":["network","langsmith","http","rate-limiting","instance-ai","cost-report"],"backgroundTag":null,"analyzedSha":"5ac6606e81f67bb9534255570cd4e86fd8101eee","analyzedAt":"2026-08-12T05:26:35.080Z","schemaVersion":2},"datasetVersion":"2026-08-13T14:17:21.547Z"}