linshenkx/prompt-optimizer · error · EvaluationValidationError

Compare test case #${index + 1} id must be unique.

Error message

Compare test case #${index + 1} id must be unique.

What it means

Thrown during compare validation when two entries in request.testCases share the same id. The service builds a Set of test case ids to map snapshots to cases; a duplicate would make the mapping ambiguous. The message names the 1-based index of the duplicate entry (Compare test case #N).

Source

Thrown at packages/core/src/services/evaluation/service.ts:501

        }
        break;

      case 'compare':
        if (!request.target?.workspacePrompt?.trim()) {
          throw new EvaluationValidationError('Workspace prompt must not be empty.');
        }
        if (!Array.isArray(request.testCases) || request.testCases.length < 1) {
          throw new EvaluationValidationError('Compare evaluation requires at least one test case.');
        }
        if (!Array.isArray(request.snapshots) || request.snapshots.length < 2) {
          throw new EvaluationValidationError('Compare evaluation requires at least two snapshots.');
        }

        const testCaseIds = new Set<string>();
        request.testCases.forEach((testCase, index) => {
          this.validateTestCase(testCase, `Compare test case #${index + 1}`);
          if (testCaseIds.has(testCase.id)) {
            throw new EvaluationValidationError(
              `Compare test case #${index + 1} id must be unique.`
            );
          }
          testCaseIds.add(testCase.id);
        });

        request.snapshots.forEach((snapshot, index) => {
          this.validateSnapshot(snapshot, `Compare snapshot #${index + 1}`);
          if (!testCaseIds.has(snapshot.testCaseId)) {
            throw new EvaluationValidationError(
              `Compare snapshot #${index + 1} references unknown testCaseId "${snapshot.testCaseId}".`
            );
          }
        });
        if (this.isImageText2ImageMode(request)) {
          const snapshotsWithMedia = request.snapshots.filter((snapshot) =>
            this.hasSnapshotOutputMedia(snapshot)
          );

View on GitHub (pinned to 3e677b1d9f)

Solutions

1. Deduplicate test cases by id before calling evaluate
2. Fix the selection logic so the same test case cannot be selected twice
3. Regenerate ids when cloning test case objects

Example fix

// before
const testCases = [tc, tc];
await svc.evaluate({ type: 'compare', testCases, snapshots, ... });

// after
const testCases = [...new Map(selected.map(tc => [tc.id, tc])).values()];
await svc.evaluate({ type: 'compare', testCases, snapshots, ... });

const ids = new Set(testCases.map(tc => tc.id));
if (ids.size !== testCases.length) throw new Error('Duplicate test case ids');
await svc.evaluate({ ...req, testCases });

const allIdsUnique = (tcs: EvaluationTestCase[]) => new Set(tcs.map(t => t.id)).size === tcs.length;

When it happens

Trigger: evaluate({type:'compare', testCases:[a, b]}) where a.id === b.id, including the same object passed twice.

Common situations: Selecting the same test case twice in a multi-select UI; concatenating test case lists that overlap without dedupe; cloning test case fixtures without regenerating ids.

Related errors

• Compare evaluation requires at least one test case.
• Compare evaluation requires at least two snapshots.
• Compare snapshot #${index + 1} references unknown testCaseId
• Image compare evaluation requires at least two snapshots wit
• Image compare evaluation requires every compared snapshot to

AI-assisted analysis of linshenkx/prompt-optimizer@3e677b1d9f (2026-08-27). Data as JSON: /api/errors/67e8f700b331f5b9. Report an issue: GitHub.