linshenkx/prompt-optimizer · error · EvaluationParseError

Evaluation result is missing a valid overall score.

Error message

Evaluation result is missing a valid overall score.

What it means

Thrown by EvaluationService when parsing an evaluation result yields no valid overall score (overall is null). The service can synthesize a minimal dimensions array from overall, but if overall itself is missing it cannot construct an EvaluationScore and aborts with EvaluationParseError.

Source

Thrown at packages/core/src/services/evaluation/service.ts:3232

        }
      }
    }

    // 如果 overall 缺失，但维度存在，则按平均分计算。
    if (overall === null && dimensions.length > 0) {
      const avg = Math.round(
        dimensions.reduce((sum, d) => sum + d.score, 0) / dimensions.length
      );
      overall = Math.max(0, Math.min(100, avg));
    }

    // 如果维度缺失，但 overall 存在，则返回一个最小维度数组。
    if (dimensions.length === 0 && overall !== null) {
      dimensions = [{ key: 'overall', label: '综合评分', score: overall }];
    }

    if (overall === null) {
      throw new EvaluationParseError('Evaluation result is missing a valid overall score.');
    }

    const score: EvaluationScore = {
      overall,
      dimensions,
    };

    // 解析 improvements（最多3条）
    const improvements = Array.isArray(data.improvements)
      ? data.improvements.map((x: any) => String(x)).filter(Boolean).slice(0, 3)
      : typeof data.improvements === 'string' && data.improvements.trim()
        ? [data.improvements.trim()].slice(0, 3)
        : [];

    // 解析 patchPlan（最多3条）
    const patchPlan = this.normalizePatchPlan(data.patchPlan || []).slice(0, 3);

    const summary = typeof data.summary === 'string' ? data.summary : '';

View on GitHub (pinned to 3e677b1d9f)

Solutions

1. Inspect the raw evaluation payload and confirm the overall score field name/format expected by the parser
2. Update the evaluation prompt to explicitly require a numeric overall score in the response schema
3. Add pre-validation of the model response before passing it to EvaluationService
4. If dimensions exist but overall is absent, compute overall yourself and pass it in

Example fix

// before
const result = await evaluationService.evaluate(rawModelOutput);
// after
const parsed = JSON.parse(rawModelOutput);
if (typeof parsed.overall !== 'number') throw new Error('model output lacks overall');
const result = await evaluationService.evaluate(rawModelOutput);

function hasOverall(raw: unknown): boolean {
  const o = typeof raw === 'string' ? JSON.parse(raw) : raw;
  return o != null && typeof (o as any).overall === 'number' && Number.isFinite((o as any).overall);
}

const isEvalPayload = (p: unknown): p is { overall: number } =>
  typeof p === 'object' && p !== null && typeof (p as any).overall === 'number';

try { await svc.evaluate(raw); } catch (e) { if (e instanceof EvaluationParseError) { /* re-prompt model or fallback */ } throw e; }

When it happens

Trigger: Calling EvaluationService's parse/evaluation API with a model response or JSON payload that contains no parseable overall score field (or a non-numeric one), e.g. an LLM returning only per-dimension scores or malformed JSON.

Common situations: LLM output format drift after a prompt/model change; evaluation prompt not instructing the model to emit an overall score; locale-specific number formats that fail numeric parsing; truncated responses.

Related errors

• Failed to parse evaluation result: no valid score JSON or re
• Result evaluation snapshot testCaseId must match testCase.id
• Image result evaluation requires at least one output image e
• Compare evaluation requires at least one test case.
• Compare evaluation requires at least two snapshots.

AI-assisted analysis of linshenkx/prompt-optimizer@3e677b1d9f (2026-08-27). Data as JSON: /api/errors/a0d463e14c426e12. Report an issue: GitHub.