linshenkx/prompt-optimizer · error · EvaluationValidationError

${label} output must not be empty.

Error message

${label} output must not be empty.

What it means

EvaluationValidationError thrown when snapshot.output is missing or blank. The output is the model response being scored, so an evaluation without output is meaningless and is rejected up front. This mirrors the other required snapshot fields (id, label, testCaseId, promptText).

Source

Thrown at packages/core/src/services/evaluation/service.ts:1733

    }
    this.validateContentBlock(testCase.input, `${label} input`);
  }

  private validateSnapshot(snapshot: EvaluationSnapshot | undefined, label: string): void {
    if (!snapshot?.id?.trim()) {
      throw new EvaluationValidationError(`${label} id must not be empty.`);
    }
    if (!snapshot?.label?.trim()) {
      throw new EvaluationValidationError(`${label} label must not be empty.`);
    }
    if (!snapshot?.testCaseId?.trim()) {
      throw new EvaluationValidationError(`${label} testCaseId must not be empty.`);
    }
    if (!snapshot?.promptText?.trim()) {
      throw new EvaluationValidationError(`${label} promptText must not be empty.`);
    }
    if (!snapshot?.output?.trim()) {
      throw new EvaluationValidationError(`${label} output must not be empty.`);
    }
    if (!snapshot?.promptRef?.kind) {
      throw new EvaluationValidationError(`${label} promptRef.kind must not be empty.`);
    }
    if (snapshot.executionInput) {
      this.validateContentBlock(snapshot.executionInput, `${label} executionInput`);
    }
    if (snapshot.outputBlock) {
      this.validateContentBlock(snapshot.outputBlock, `${label} outputBlock`);
    }
  }

  private normalizeContentBlock(block?: EvaluationContentBlock): NormalizedContentBlock | undefined {
    const label = block?.label?.trim() || '';
    const content = block?.content?.trim() || '';
    if (!label || (!content && !this.hasBlockMedia(block))) {
      return undefined;
    }

View on GitHub (pinned to 3e677b1d9f)

Solutions

1. Skip or queue snapshots whose upstream model call produced empty output
2. Populate snapshot.output with the model's final text before calling the evaluation API
3. Add an assertion after the model call: if (!output.trim()) skip/flag the case

Example fix

// before
snaps.push({ ...base, output: res.output ?? '' });

// after
if (res.output?.trim()) snaps.push({ ...base, output: res.output });
else skipped.push(base.testCaseId);

if (!modelOutput?.trim()) { skipped.push(testCaseId); continue; } // never evaluate empty outputs

const hasOutput = (s: unknown): s is { output: string } => typeof (s as any)?.output === 'string' && (s as any).output.trim().length > 0;

When it happens

Trigger: Submitting a snapshot whose output field is undefined, null, empty, or whitespace — e.g. the model call failed or returned empty content and the caller proceeded to evaluate anyway.

Common situations: Evaluating failed/aborted LLM calls without checking the response body first; streaming pipelines that persist snapshots before the full output arrives; accidentally mapping response.text to the wrong key.

Related errors

• Result evaluation snapshot testCaseId must match testCase.id
• Image result evaluation requires at least one output image e
• Compare evaluation requires at least one test case.
• Compare evaluation requires at least two snapshots.
• Compare test case #${index + 1} id must be unique.

AI-assisted analysis of linshenkx/prompt-optimizer@3e677b1d9f (2026-08-27). Data as JSON: /api/errors/2eca7e37b6ad4314. Report an issue: GitHub.