n8n-io/n8n · error · Error

LangSmith runs/query failed: ${res.status} ${await res.text(

Error message

LangSmith runs/query failed: ${res.status} ${await res.text()}

What it means

Thrown by sumThreadCost() (build-cost-report.ts:171) when the LangSmith /api/v1/runs/query POST returns a non-OK status after the lsFetch backoff. This query joins a specific thread_id to its root runs (one per build turn) within the resolved session/project to sum total_cost and total_tokens. Like the sessions lookup, it has 429/5xx backoff, so this error indicates a hard failure or exhausted retries. The response status and body are echoed for diagnosis.

Source

Thrown at packages/@n8n/instance-ai/evaluations/cli/build-cost-report.ts:171

 *  --trace-project, or the trace hasn't landed yet). */
export async function sumThreadCost(
	ls: LangSmithConfig,
	projectId: string,
	threadId: string,
): Promise<ThreadCost | null> {
	const res = await lsFetch(`${ls.apiUrl}/api/v1/runs/query`, {
		method: 'POST',
		headers: ls.headers,
		body: JSON.stringify({
			session: [projectId],
			is_root: true,
			filter: `eq(thread_id, "${threadId}")`,
			limit: RUNS_QUERY_LIMIT,
			select: ['id', 'total_cost', 'total_tokens'],
		}),
	});
	if (!res.ok) {
		throw new Error(`LangSmith runs/query failed: ${res.status} ${await res.text()}`);
	}
	const { runs } = lsRunsQuerySchema.parse(await res.json());
	if (runs.length === 0) return null;
	if (runs.length === RUNS_QUERY_LIMIT) {
		console.warn(
			`thread ${threadId}: hit the runs/query page limit (${RUNS_QUERY_LIMIT}) — cost may be undercounted`,
		);
	}
	return {
		costUsd: runs.reduce((sum, r) => sum + (r.total_cost ?? 0), 0),
		tokens: runs.reduce((sum, r) => sum + (r.total_tokens ?? 0), 0),
		turns: runs.length,
	};
}

// ---------------------------------------------------------------------------
// Per-case aggregation
// ---------------------------------------------------------------------------

View on GitHub (pinned to 5ac6606e81)

Solutions

Retry the command — transient 5xx/429 storms often clear on the next run.
Lower --concurrency to reduce rate-limit pressure on the runs/query endpoint (default 3).
For 401/403, regenerate LANGSMITH_API_KEY or check its scopes.
If a specific thread_id is malformed, inspect the eval-results.json threadIds array for corruption.
Check the LangSmith status page for ongoing incidents.

Example fix

// before (high concurrency tripping rate limits)
pnpm tsx evaluations/cli/build-cost-report.ts --results run/eval-results.json --concurrency 16
// after
pnpm tsx evaluations/cli/build-cost-report.ts --results run/eval-results.json --concurrency 3

Defensive patterns

Strategy: retry

Validate before calling

// Lower concurrency to avoid rate limits; the built-in lsFetch already retries 429/5xx.
// Pass --concurrency 1 or 2 when joining many threads.
const concurrency = Math.min(desiredConcurrency, 2);

Try / catch

// Retry the whole report on transient runs/query failures (lsFetch already retries per-request):
for (let attempt = 1; attempt <= 3; attempt++) {
  try {
    await runReport(args);
    break;
  } catch (error) {
    const msg = error instanceof Error ? error.message : String(error);
    if (msg.includes('runs/query failed') && attempt < 3) {
      await new Promise((r) => setTimeout(r, 10_000 * attempt));
      continue;
    }
    throw error;
  }
}

Prevention

Use a low --concurrency (1-3) for large thread joins to stay under rate limits.
Retry the full report on transient 5xx — lsFetch's per-request backoff is bounded.
Monitor LangSmith status page during large joins.
Ensure LANGSMITH_API_KEY has runs/query scope.

When it happens

Trigger: During a full-run thread join (threadJoinCosts), each thread fires a runs/query. A single thread's query hitting a hard 4xx (auth, permissions, malformed filter) or a 5xx that exhausted 6 backoff attempts trips this. A full-run join fires ~100 queries, increasing the chance one trips the rate limit and exhausts backoff.

Common situations: Rate-limiting (429) that exhausted the 6-attempt exponential backoff, a transient 5xx storm, an auth key that was valid for sessions but scoped/restricted for runs/query, or a malformed thread_id producing a 4xx. High-concurrency joins (--concurrency large) can amplify rate-limit pressure.

Related errors

AI-assisted analysis of n8n-io/n8n@5ac6606e81 (2026-08-12). Data as JSON: /api/errors/5759f6fb851cd65c. Report an issue: GitHub.