What causes "formatExecutionErrorMessage(error)" in linshenkx/prompt-optimizer?

Inspect the wrapped message and the original cause (error.cause) to identify the real failure

linshenkx/prompt-optimizer · error · EvaluationExecutionError

formatExecutionErrorMessage(error)

Error message

formatExecutionErrorMessage(error)

What it means

EvaluationExecutionError wrapping any underlying failure thrown while invoking the judge/synthesis model during an evaluation run. The original error message is formatted via formatExecutionErrorMessage and the original Error is preserved as the cause. It signals an infrastructure/transport/model failure rather than bad input data.

Source

Thrown at packages/core/src/services/evaluation/service.ts:2202

      const synthesisResult = await this.llmService.sendMessage(
        synthesisMessages,
        request.evaluationModelKey
      );
      const duration = Date.now() - startTime;
      const responseMetadata = this.buildResponseMetadata(
        request,
        {
          model: request.evaluationModelKey,
          timestamp: Date.now(),
          duration,
          compareJudgements: judgeResults,
        },
        normalizedCompare
      );

      return this.parseEvaluationResult(synthesisResult, request.type, responseMetadata);
    } catch (error) {
      throw new EvaluationExecutionError(
        this.formatExecutionErrorMessage(error),
        error instanceof Error ? error : undefined
      );
    }
  }

  private async evaluateStructuredCompareStream(
    request: Extract<EvaluationRequest, { type: 'compare' }>,
    normalizedCompare: NormalizedCompareContext,
    callbacks: EvaluationStreamHandlers
  ): Promise<void> {
    const startTime = Date.now();
    const language = await this.resolveComparePromptLanguage();
    const subject = this.resolveComparePromptSubjectConfig(request.mode, language);

    try {
      const judgeResults = await this.executeStructuredCompareJudgePlan(
        request,

View on GitHub (pinned to 3e677b1d9f)

Solutions

Inspect the wrapped message and the original cause (error.cause) to identify the real failure
Fix provider credentials/rate limits/network issues accordingly
Retry transient failures with exponential backoff; cap promptText length before evaluating

Example fix

// before
const result = await svc.runEvaluation(req);

// after
try {
  const result = await svc.runEvaluation(req);
} catch (e) {
  if (e instanceof EvaluationExecutionError) {
    logger.error('eval failed', { cause: e.cause?.message });
    await backoff.retry(() => svc.runEvaluation(req));
  } else throw e;
}

Defensive patterns

Strategy: retry

Validate before calling

await provider.models.list(); // cheap pre-flight to verify credentials/connectivity before a long eval run

Try / catch

try {
  const result = await svc.runEvaluation(req);
} catch (e) {
  if (e instanceof EvaluationExecutionError) {
    const cause = (e.cause as Error)?.message ?? e.message;
    if (isTransient(cause)) return await retry(() => svc.runEvaluation(req), { retries: 3, backoff: 'exponential' });
    throw e;
  }
  throw e;
}

Prevention

Verify API keys and provider connectivity before batch runs
Set sane timeouts and cap prompt sizes
Use exponential-backoff retry for rate-limit/network causes

When it happens

Trigger: Any exception from the model call or comparison step inside the evaluation execution path: network timeout, provider 4xx/5xx, rate limiting, invalid API key, or malformed request to the model provider.

Common situations: Expired or wrong LLM API key; provider rate limits during batch evaluations; proxy/firewall blocking the model endpoint; oversized promptText exceeding the provider's context limit mid-run.

Related errors

AI-assisted analysis of linshenkx/prompt-optimizer@3e677b1d9f (2026-08-27). Data as JSON: /api/errors/176072b4cb3b473a. Report an issue: GitHub.