linshenkx/prompt-optimizer · critical · EvaluationExecutionError

Image understanding service is not available for multimodal

Error message

Image understanding service is not available for multimodal evaluation.

What it means

EvaluationExecutionError thrown by executeMultimodalEvaluation when shouldUseMultimodalEvaluation(request) is true (image-mode result/compare) but no imageUnderstandingService was provided to EvaluationService. Multimodal evaluation requires that dependency to send images to a vision-capable model; without it the request cannot run.

Source

Thrown at packages/core/src/services/evaluation/service.ts:705

    if (request.type === 'result') {
      return this.hasSnapshotOutputMedia(request.snapshot);
    }

    if (request.type === 'compare') {
      return request.snapshots.some((snapshot) => this.hasSnapshotOutputMedia(snapshot));
    }

    return false;
  }

  private async executeMultimodalEvaluation(
    request: Extract<EvaluationRequest, { type: 'result' | 'compare' }>,
    messages: Message[],
    modelConfig: TextModelConfig
  ): Promise<string> {
    if (!this.imageUnderstandingService) {
      throw new EvaluationExecutionError(
        'Image understanding service is not available for multimodal evaluation.'
      );
    }

    const { systemPrompt, userPrompt } = this.splitEvaluationMessages(messages);
    const resolvedMedia = await this.resolveEvaluationMedia(request);
    const manifest = this.buildImageEvidenceManifest(resolvedMedia);

    const result = await this.imageUnderstandingService.understand({
      modelConfig,
      systemPrompt,
      userPrompt: `${userPrompt}\n\n${manifest}`.trim(),
      images: resolvedMedia.map((item) => ({
        b64: item.b64,
        mimeType: item.mimeType,
      })),
    });

View on GitHub (pinned to 3e677b1d9f)

Solutions

1. Provide an imageUnderstandingService instance when constructing EvaluationService
2. If you don't need image evaluation, change mode.functionMode/subMode so multimodal path isn't selected
3. Add a startup assertion that imageUnderstandingService is present whenever image modes are configurable

Example fix

// before
const svc = new EvaluationService({ modelManager, templateManager, llmService });

// after
const svc = new EvaluationService({
  modelManager, templateManager, llmService,
  imageUnderstandingService: new ImageUnderstandingService(...),
});

if (isImageMode(req.mode) && !imageUnderstandingService) {
  throw new Error('Image evaluation requires imageUnderstandingService to be wired');
}

const canDoMultimodal = (svc: EvaluationService) => Boolean((svc as any).imageUnderstandingService);

try { await svc.evaluateStream(req, cb); } catch (e) { if (e instanceof EvaluationExecutionError && /Image understanding service/.test(e.message)) disableImageModes(); else throw e; }

When it happens

Trigger: Constructing EvaluationService without imageUnderstandingService, then calling evaluate/evaluateStream with an image text-2-image request that passes validateRequest and reaches executeMultimodalEvaluation (invoked from evaluateStream's content path).

Common situations: DI wiring incomplete in tests or a trimmed-down deployment; new image evaluation feature enabled by mode config while the service composition was never updated; optional dependency omitted because text evaluations worked fine.

Related errors

• Image storage service is required to resolve evaluation imag
• Image result evaluation requires at least one output image e
• No valid image evidence could be resolved for evaluation.
• Result evaluation snapshot testCaseId must match testCase.id
• Compare evaluation requires at least one test case.

AI-assisted analysis of linshenkx/prompt-optimizer@3e677b1d9f (2026-08-27). Data as JSON: /api/errors/f06364e5c23b8439. Report an issue: GitHub.