n8n-io/n8n · error · Error

suite "${args.suite}" not found. Available: ${known || '(non

Error message

suite "${args.suite}" not found. Available: ${known || '(none)'}.

What it means

After parsing args, langtracer-push calls client.listSuites() to fetch all suites from the lang-tracer server (line 168), then looks for a match by slug or numeric ID (line 169). If no suite matches, it throws with the sorted list of available suite slugs to help the user pick the correct one. This requires valid LANGTRACER_URL and LANGTRACER_API_KEY env vars, resolved by resolveLangTracerConfig before the client is constructed.

Source

Thrown at packages/@n8n/instance-ai/evaluations/cli/langtracer-push.ts:175

async function main() {
	const parsed = parseArgs(process.argv.slice(2));
	if (parsed.helpRequested) {
		console.log(HELP);
		return;
	}
	const args = parsed.args;

	const client = new LangTracerClient(resolveLangTracerConfig());

	const suites = await client.listSuites();
	const suite = suites.find((s) => s.slug === args.suite || String(s.id) === args.suite);
	if (!suite) {
		const known = suites
			.map((s) => s.slug)
			.sort()
			.join(', ');
		throw new Error(`suite "${args.suite}" not found. Available: ${known || '(none)'}.`);
	}

	// Select disk cases: loader applies --filter/--exclude, --tier narrows by the
	// case's datasets (mirrors data/source.ts); then narrow to the exact slugs
	// from positional args + --changed (if either was given).
	const loaded = [
		...loadWorkflowTestCasesWithFiles(args.filter, args.exclude),
		...loadAgentEvalTestCasesWithFiles(args.filter, args.exclude),
	];
	const dupes = loaded.filter((c, i) => loaded.findIndex((o) => o.fileSlug === c.fileSlug) !== i);
	if (dupes.length > 0) {
		throw new Error(
			`duplicate case slug(s) across data/workflows and data/agents: ${dupes.map((d) => d.fileSlug).join(', ')}`,
		);
	}
	const tier = args.tier;
	const all = tier ? loaded.filter((c) => c.testCase.datasets.includes(tier)) : loaded;
	const exactSlugs = new Set([...args.slugs, ...(args.changed ? gitChangedSlugs() : [])]);

View on GitHub (pinned to 5ac6606e81)

Solutions

1. Check the 'Available:' list in the error message for valid suite slugs
2. Verify the suite slug spelling against the listed available suites
3. Create the suite in the lang-tracer UI first if none exists
4. Confirm LANGTRACER_URL points at the correct lang-tracer instance

Example fix

# before
pnpm eval:langtracer-push --suite basline --changed  # typo

# after
pnpm eval:langtracer-push --suite baseline --changed

// Pre-check suite existence before running the full push:
const suites = await client.listSuites();
const suite = suites.find(s => s.slug === args.suite || String(s.id) === args.suite);
if (!suite) {
  const known = suites.map(s => s.slug).sort().join(', ');
  throw new Error(`suite "${args.suite}" not found. Available: ${known || '(none)'}.`);
}

try {
  const suites = await client.listSuites();
  const suite = suites.find(s => s.slug === args.suite || String(s.id) === args.suite);
  if (!suite) throw new Error(`suite not found`);
} catch (error) {
  // Network errors vs not-found: surface the available suites list for diagnostics
  console.error(error instanceof Error ? error.message : error);
  process.exit(1);
}

When it happens

Trigger: Running `--suite baseline` when no suite named 'baseline' exists on the lang-tracer server, or `--suite 999` when no suite has that numeric ID. Also fires if the API key lacks access to the suite, or if LANGTRACER_URL points at the wrong instance.

Common situations: Typo in the suite slug. The suite was renamed or deleted on the server. Wrong lang-tracer environment (staging vs production). The API key does not have access to the target suite. Fresh lang-tracer instance with no suites created yet.

Related errors

• --source langtracer requires --suite <slug>
• --suite <slug|id> is required
• --set-kind must be "regression" or "capability_gap"
• Unknown flag: ${arg.split('=', 1)[0]} (use --help)
• select cases to push: pass <slugs...>, --changed, --filter,

AI-assisted analysis of n8n-io/n8n@5ac6606e81 (2026-08-12). Data as JSON: /api/errors/2842e223c09f881a. Report an issue: GitHub.