linshenkx/prompt-optimizer · error · EvaluationValidationError

${label} testCaseId must not be empty.

Error message

${label} testCaseId must not be empty.

What it means

Thrown by EvaluationService when validating an evaluation snapshot whose testCaseId is missing or whitespace-only. It is part of a chain of required-field checks on snapshot objects before an evaluation can be created or executed. The error is an EvaluationValidationError, meaning caller-supplied input failed validation rather than an internal failure.

Source

Thrown at packages/core/src/services/evaluation/service.ts:1727

    }
  }

  private validateTestCase(testCase: EvaluationTestCase | undefined, label: string): void {
    if (!testCase?.id?.trim()) {
      throw new EvaluationValidationError(`${label} id must not be empty.`);
    }
    this.validateContentBlock(testCase.input, `${label} input`);
  }

  private validateSnapshot(snapshot: EvaluationSnapshot | undefined, label: string): void {
    if (!snapshot?.id?.trim()) {
      throw new EvaluationValidationError(`${label} id must not be empty.`);
    }
    if (!snapshot?.label?.trim()) {
      throw new EvaluationValidationError(`${label} label must not be empty.`);
    }
    if (!snapshot?.testCaseId?.trim()) {
      throw new EvaluationValidationError(`${label} testCaseId must not be empty.`);
    }
    if (!snapshot?.promptText?.trim()) {
      throw new EvaluationValidationError(`${label} promptText must not be empty.`);
    }
    if (!snapshot?.output?.trim()) {
      throw new EvaluationValidationError(`${label} output must not be empty.`);
    }
    if (!snapshot?.promptRef?.kind) {
      throw new EvaluationValidationError(`${label} promptRef.kind must not be empty.`);
    }
    if (snapshot.executionInput) {
      this.validateContentBlock(snapshot.executionInput, `${label} executionInput`);
    }
    if (snapshot.outputBlock) {
      this.validateContentBlock(snapshot.outputBlock, `${label} outputBlock`);
    }
  }

View on GitHub (pinned to 3e677b1d9f)

Solutions

1. Set snapshot.testCaseId to a non-empty, trimmed string identifier of the test case
2. Log the offending snapshot before calling the service to spot missing fields early
3. Add a runtime schema check (e.g. zod) on incoming payload shapes before calling EvaluationService

Example fix

// before
const snapshot = { label: 'case A', promptText: '...', output: '...' };
await svc.createEvaluation({ ...snapshot });

// after
const snapshot = { label: 'case A', testCaseId: testCase.id, promptText: '...', output: '...' };
await svc.createEvaluation({ ...snapshot });

function hasSnapshotRequiredFields(s: any): boolean {
  return Boolean(s?.testCaseId?.trim()) && Boolean(s?.label?.trim()) && Boolean(s?.promptText?.trim()) && Boolean(s?.output?.trim());
}
const invalid = snapshots.filter(s => !hasSnapshotRequiredFields(s));
if (invalid.length) throw new Error(`Missing required snapshot fields on ${invalid.length} items`);

function isEvaluableSnapshot(s: unknown): s is { testCaseId: string; label: string; promptText: string; output: string; promptRef: { kind: string } } {
  const v = s as any;
  return typeof v?.testCaseId === 'string' && v.testCaseId.trim() !== ''
    && typeof v?.label === 'string' && v.label.trim() !== ''
    && typeof v?.promptText === 'string' && v.promptText.trim() !== ''
    && typeof v?.output === 'string' && v.output.trim() !== ''
    && typeof v?.promptRef?.kind === 'string' && v.promptRef.kind !== '';
}

try { await svc.createEvaluation(req); } catch (e) { if (e instanceof EvaluationValidationError) { /* fix input fields, report to user */ } else throw e; }

When it happens

Trigger: Calling an EvaluationService API that validates snapshots (e.g. createEvaluation / run evaluation with testCase snapshots) where snapshot.testCaseId is undefined, empty string, or only whitespace after trim().

Common situations: Building snapshots from loose JSON payloads or CSV imports where the test-case id column is absent; renaming fields (case_id vs testCaseId) after a model/type change; constructing snapshots programmatically and forgetting to propagate the parent test case id.

Related errors

• Result evaluation snapshot testCaseId must match testCase.id
• Image result evaluation requires at least one output image e
• Compare evaluation requires at least one test case.
• Compare evaluation requires at least two snapshots.
• Compare test case #${index + 1} id must be unique.

AI-assisted analysis of linshenkx/prompt-optimizer@3e677b1d9f (2026-08-27). Data as JSON: /api/errors/8dde4af60707f48b. Report an issue: GitHub.