linshenkx/prompt-optimizer · error · EvaluationValidationError

Evaluation model key must not be empty.

Error message

Evaluation model key must not be empty.

What it means

OptimizationError thrown by optimizePrompt when modelManager.getModel(request.modelKey) resolves to nothing — the requested model key is not in the model registry. It fails before any LLM call, meaning the model was deleted, never existed, or the key is misspelled.

Source

Thrown at packages/core/src/services/evaluation/service.ts:454

    try {
      await this.llmService.sendMessageStream(messages, request.evaluationModelKey, streamHandlers);
    } catch (error) {
      callbacks.onError(
        new EvaluationExecutionError(
          this.formatExecutionErrorMessage(error),
          error instanceof Error ? error : undefined
        )
      );
    }
  }

  /**
   * 验证评估请求
   */
  private validateRequest(request: EvaluationRequest): void {
    if (!request.evaluationModelKey?.trim()) {
      throw new EvaluationValidationError('Evaluation model key must not be empty.');
    }

    if (!request.mode) {
      throw new EvaluationValidationError('Evaluation mode configuration must not be empty.');
    }
    if (!request.mode.functionMode) {
      throw new EvaluationValidationError('Function mode must not be empty.');
    }
    if (!request.mode.subMode) {
      throw new EvaluationValidationError('Sub mode must not be empty.');
    }

    switch (request.type) {
      case 'result':
        if (!request.target?.workspacePrompt?.trim()) {
          throw new EvaluationValidationError('Workspace prompt must not be empty.');
        }
        this.validateTestCase(request.testCase, 'Result evaluation test case');

View on GitHub (pinned to 3e677b1d9f)

Solutions

1. Verify the key exists first: (await modelManager.listModels()).some(m => m.key === request.modelKey).
2. Re-point the request at an existing model or re-add the missing model configuration.
3. Ensure model loading completes (await initialization) before optimization calls.
4. Fall back to a known-good default model when the requested one is absent.

Example fix

// before
await promptService.optimizePrompt({ modelKey: savedKey, targetPrompt: p });

// after
const models = await modelManager.listModels();
const key = models.some(m => m.key === savedKey) ? savedKey : models[0]?.key;
if (!key) throw new Error('No models configured');
await promptService.optimizePrompt({ modelKey: key, targetPrompt: p });

const models = await modelManager.listModels();
if (!models.some(m => m.key === request.modelKey)) throw new Error('Model not configured');

async function modelExists(modelManager: any, key: string): Promise<boolean> {
  return !!(await modelManager.getModel(key));
}

try { await svc.optimizePrompt(req); } catch (e) { if (e instanceof OptimizationError && e.message === 'Model not found') { await refreshModels(); } }

When it happens

Trigger: Calling optimizePrompt with a key not present in the model manager — deleted model, typo, key from another environment/profile, or models not yet loaded when the call runs.

Common situations: User deletes a model that was set as default; settings imported from another machine referencing missing models; model list loads async and the call races it; key casing or provider-prefix mistakes ('openai/gpt-4' vs 'gpt-4').

Related errors

• Function mode must not be empty.
• Model ${provider} not found
• Data must be an object
• Evaluation mode configuration must not be empty.
• Sub mode must not be empty.

AI-assisted analysis of linshenkx/prompt-optimizer@3e677b1d9f (2026-08-27). Data as JSON: /api/errors/9fa09f8a2d5df59b. Report an issue: GitHub.