n8n-io/n8n · error · Error

LangSmith mode requires dataset to be a dataset name string

Error message

LangSmith mode requires dataset to be a dataset name string

What it means

The LangSmith runner expects `dataset` to be the dataset NAME (a string) so it can call `readDataset({datasetName})`. If a non-string is passed (object, number, array), the runner refuses to proceed because the underlying API call would fail with an opaque error. This is a defensive type check at the boundary of the LangSmith execution path.

Source

Thrown at packages/@n8n/ai-workflow-builder.ee/evaluations/harness/runner.ts:1591

				feedback,
				durationMs: totalDurationMs,
				generationDurationMs: genDurationMs,
				workflow,
				error: errorMessage,
			};

			artifactSaver?.saveExample(result);
			capturedResults.push(result);
			lifecycle?.onExampleComplete?.(index, result);

			return { workflow, prompt, feedback };
		}
	};

	const feedbackExtractor = createLangsmithFeedbackExtractor();

	if (typeof dataset !== 'string') {
		throw new Error('LangSmith mode requires dataset to be a dataset name string');
	}

	const data = await resolveAndEnrichLangsmithData({ dataset, langsmithOptions, lsClient, logger });

	const effectiveData = applyRepetitions(data, langsmithOptions.repetitions);

	totalExamples = Array.isArray(effectiveData) ? effectiveData.length : 0;

	logLangsmithInputsSummary(logger, effectiveData);
	const { experimentName, experimentId, datasetId } = await runLangsmithEvaluateAndFlush({
		target,
		effectiveData,
		feedbackExtractor,
		langsmithOptions,
		lsClient,
		logger,
		targetCallCount: () => targetCallCount,
	});

View on GitHub (pinned to 5ac6606e81)

Solutions

Pass the dataset NAME string: `runLangsmithEvaluation({ dataset: 'my-dataset', ... })`.
If you only have the dataset object/id, resolve its name first and pass that.
Type the parameter at the call site as `dataset: string` so the misuse is caught at compile time.

Example fix

// before
runLangsmithEvaluation({ dataset: { name: 'my-dataset' }, ... });
// after
runLangsmithEvaluation({ dataset: 'my-dataset', ... });

Defensive patterns

Strategy: type-guard

Validate before calling

function assertDatasetName(dataset: unknown): asserts dataset is string {
  if (typeof dataset !== 'string' || dataset.length === 0) {
    throw new Error('dataset must be a non-empty dataset-name string');
  }
}
assertDatasetName(options.dataset);

Type guard

function isDatasetName(v: unknown): v is string {
  return typeof v === 'string' && v.length > 0;
}

Prevention

Type the option as `dataset: string` at every call site so misuses are compile errors.
If you carry dataset objects around, resolve the name at the boundary and pass only the string downstream.
Add a unit test that passes an object and asserts the type guard rejects it.

When it happens

Trigger: Programmatic callers passing the dataset object directly (`{id: '...'} ` or the resolved dataset) instead of its name, or a config builder that spreads an object into the field. The `typeof dataset !== 'string'` guard trips.

Common situations: Refactor that changed the `dataset` parameter from a name to an object without updating call sites; misuse from a wrapper script; a default-config object leaking into the field.

Related errors

AI-assisted analysis of n8n-io/n8n@5ac6606e81 (2026-08-12). Data as JSON: /api/errors/821249fa9c902489. Report an issue: GitHub.