{"record":{"id":"a51440a8257cbb97","repo":"linshenkx/prompt-optimizer","slug":"failed-to-parse-evaluation-result-no-valid-score","errorCode":null,"errorMessage":"Failed to parse evaluation result: no valid score JSON or recognizable overall score found. Raw content length: ${content.length} characters.","messagePattern":"Failed to parse evaluation result: no valid score JSON or recognizable overall score found\\. Raw content length: (.+?) characters\\.","errorType":"exception","errorClass":"EvaluationParseError","httpStatus":null,"severity":"error","filePath":"packages/core/src/services/evaluation/service.ts","lineNumber":2975,"sourceCode":"\n        const normalized = this.normalizeEvaluationResponse(payload as any, type, metadata);\n        return normalized;\n      } catch (e) {\n        console.warn(\n          '[EvaluationService] Failed to parse evaluation JSON candidate:',\n          e instanceof Error ? e.message : String(e)\n        );\n      }\n    }\n\n    // 降级解析\n    const textResult = this.parseTextEvaluation(content, type, metadata);\n    if (textResult) {\n      console.warn('[EvaluationService] Using text fallback parsing');\n      return textResult;\n    }\n\n    throw new EvaluationParseError(\n      `Failed to parse evaluation result: no valid score JSON or recognizable overall score found. Raw content length: ${content.length} characters.`\n    );\n  }\n\n  /**\n   * 从模型输出中提取可能的 JSON 片段。\n   *\n   * 现实中模型可能：\n   * - 输出 ```json ... ```\n   * - 输出 ``` ... ```（无语言标注）\n   * - 在解释文字中夹杂一段 JSON\n   */\n  private extractJsonCandidates(content: string): string[] {\n    const candidates: string[] = [];\n\n    // 1) 优先提取所有 fenced code block（不限语言），只挑看起来像 JSON 的块。\n    const fencedRegex = /```[a-zA-Z0-9_-]*\\s*([\\s\\S]*?)\\s*```/g;\n    for (const match of content.matchAll(fencedRegex)) {","sourceCodeStart":2957,"sourceCodeEnd":2993,"githubUrl":"https://github.com/linshenkx/prompt-optimizer/blob/3e677b1d9f7e0493c142c175560531e7ae786dce/packages/core/src/services/evaluation/service.ts#L2957-L2993","documentation":"EvaluationParseError thrown after both structured JSON parsing and text-fallback parsing fail to find any score in the judge model's output. The service first tries to extract score JSON, then a text fallback (parseTextEvaluation); if neither yields a recognizable overall score, this error surfaces with the raw content length for diagnosis.","triggerScenarios":"The judge model returns prose, refusal text, or malformed JSON with no parseable score field; or the output format drifted so neither the JSON extractor nor the regex/heuristic text parser recognizes an overall score.","commonSituations":"Switching judge models that ignore the output-format instructions; prompt-injection or safety refusals returning 'I cannot evaluate...'; truncated responses hitting max_tokens before the score is emitted; non-English outputs the text parser doesn't recognize.","solutions":["Inspect the raw model content (log it where the error is caught) to see what was actually returned","Strengthen the judge prompt to require strict JSON with an overall score, or increase max_tokens to avoid truncation","Pin or switch to a judge model known to follow the format; retry once — occasional malformed outputs are common"],"exampleFix":"// before\nconst result = await svc.runEvaluation(req);\n\n// after\nlet result;\ntry { result = await svc.runEvaluation(req); }\ncatch (e) {\n  if (e instanceof EvaluationParseError) result = await svc.runEvaluation({ ...req, temperature: 0 });\n  else throw e;\n}","handlingStrategy":"retry","validationCode":"null","typeGuard":null,"tryCatchPattern":"try { result = await svc.runEvaluation(req); }\ncatch (e) {\n  if (e instanceof EvaluationParseError) {\n    logRawContentForDebugging();\n    result = await svc.runEvaluation({ ...req, judgePrompt: stricterJsonPrompt, temperature: 0 });\n  } else throw e;\n}","preventionTips":["Instruct the judge model to output strict JSON with an overall score","Set max_tokens high enough to avoid truncation before the score appears","Log raw judge outputs during development to catch format drift early"],"tags":["evaluation","parsing","llm-output"],"backgroundTag":"llm-output-parse-failed","analyzedSha":"3e677b1d9f7e0493c142c175560531e7ae786dce","analyzedAt":"2026-08-27T21:29:16.709Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}