langfuse/langfuse · error · LangfuseConflictError

Multiple evaluators named "${input.name}" exist in this proj

Error message

Multiple evaluators named "${input.name}" exist in this project

What it means

upsertByName looks up evaluators by project + name and throws LangfuseConflictError when more than one match exists, because name-based upsert is only safe when the name unambiguously identifies a single evaluator.

Source

Thrown at web/src/features/evals/v2/server/evaluators/evaluatorService.ts:299

    });
    return evaluator;
  }

  // Temporary fallback for the unstable Evaluators API until the final API
  // exposes explicit create and update semantics.
  async upsertByName(
    input: CreateEvaluatorInput,
    createdByUserId: string | null,
  ) {
    const block = await validateEvaluatorForPersistence(input);
    const result = await this.prisma.$transaction(async (prisma) => {
      const matches = await repository.findEvaluatorsByName({
        prisma,
        projectId: input.projectId,
        name: input.name,
      });
      if (matches.length > 1) {
        throw new LangfuseConflictError(
          `Multiple evaluators named "${input.name}" exist in this project`,
        );
      }

      const existing = matches[0];
      if (!existing) {
        return {
          action: "create" as const,
          evaluator: await repository.createEvaluator({
            prisma,
            input: {
              ...input,
              definition: prepareEvaluatorDefinitionForPersistence(
                input.definition,
              ),
            },
            createdByUserId,
            block,

View on GitHub (pinned to 59d92c7cf3)

Solutions

1. Rename or delete the duplicate evaluators so the name is unique within the project, then retry the upsert
2. Prefer id-based create/update when duplicate names are expected
3. Guard UI flows that rename evaluators from enforcing name uniqueness

Example fix

// before
await evaluatorService.upsertByName({ projectId, name: "accuracy", ... });
// after
const dupes = await repository.findEvaluatorsByName({ prisma, projectId, name: "accuracy" });
if (dupes.length > 1) throw new Error("Resolve duplicate evaluators named 'accuracy' first");
await evaluatorService.upsertByName({ projectId, name: "accuracy", ... });

const matches = await repository.findEvaluatorsByName({ prisma, projectId, name: input.name });
if (matches.length > 1) throw new Error(`Resolve ${matches.length} duplicates named "${input.name}" before upsert`);

null

try { await evaluatorService.upsertByName(input); } catch (e) { if (e instanceof LangfuseConflictError && e.message.startsWith("Multiple evaluators")) { /* list duplicates for the user to rename/delete */ } else throw e; }

When it happens

Trigger: Calling upsertByName (create/update evaluator by name) when the project already contains two or more evaluators sharing that name — usually from before a unique-name constraint existed, or created via direct API with explicit distinct ids.

Common situations: Historic data with duplicate names in one project; migrating to name-based upsert on an old project; concurrent creation of same-named evaluators via id-based create.

Related errors

• An evaluator named "${input.name}" already exists with a dif
• CONFLICT
• buildTemplateInUseMessage(${names})
• An evaluator with this id already exists
• conflict

AI-assisted analysis of langfuse/langfuse@59d92c7cf3 (2026-08-27). Data as JSON: /api/errors/3a76d4b1c618c8a0. Report an issue: GitHub.