linshenkx/prompt-optimizer · error · VariableValueGenerationParseError

Failed to parse LLM response: ${error instanceof Error ? err

Error message

Failed to parse LLM response: ${error instanceof Error ? error.message : String(error)}

What it means

parseGenerationResult first tries structured extraction, then falls back to raw JSON.parse on the model's text; if both fail, VariableValueGenerationParseError is thrown carrying the FIRST (structured-extraction) error message. It means the LLM's output is not parseable JSON at all.

Source

Thrown at packages/core/src/services/variable-value-generation/service.ts:186

    // 1. 尝试提取 JSON 代码块
    const jsonMatch = textContent.match(/```json\s*([\s\S]*?)\s*```/i);
    const jsonText = jsonMatch ? jsonMatch[1] : textContent;

    try {
      // 2. 使用 jsonrepair 修复可能的格式问题
      const repaired = jsonrepair(jsonText);
      const parsed = JSON.parse(repaired);

      // 3. 标准化响应（传递请求的变量列表用于对齐）
      return this.normalizeGenerationResponse(parsed, requestedVariables);
    } catch (error) {
      // 回退：尝试直接解析
      try {
        const parsed = JSON.parse(jsonText);
        return this.normalizeGenerationResponse(parsed, requestedVariables);
      } catch (fallbackError) {
        throw new VariableValueGenerationParseError(
          `Failed to parse LLM response: ${error instanceof Error ? error.message : String(error)}`
        );
      }
    }
  }

  /**
   * 标准化并验证生成响应
   * 🔧 修复：添加变量对齐校验，确保返回的变量与请求一致
   */
  private normalizeGenerationResponse(
    data: any,
    requestedVariables: VariableToGenerate[]
  ): VariableValueGenerationResponse {
    if (!data || typeof data !== 'object') {
      throw new VariableValueGenerationParseError('Generation result is not a valid object.');
    }

View on GitHub (pinned to 3e677b1d9f)

Solutions

Log the raw model output to see exactly why JSON.parse failed
Raise max_tokens so the response is not truncated
Strengthen the template to demand raw JSON only — no markdown fences, no commentary
Strip fences/commentary before parsing if you control the pipeline, or retry once
Switch to a model with reliable JSON-mode/structured-output support

Example fix

// before
const res = await gen.generate(req);

// after
try {
  const res = await gen.generate(req);
} catch (e) {
  if (e instanceof VariableValueGenerationParseError) {
    const res2 = await gen.generate({ ...req, /* raise token budget / stricter template */ });
    return res2;
  }
  throw e;
}

Defensive patterns

Strategy: retry

Validate before calling

const looksLikeJson = (t: string) => { const s = t.trim(); return s.startsWith('{') || s.startsWith('['); };

Try / catch

catch (e) { if (e instanceof VariableValueGenerationParseError) { return gen.generate(req); /* one bounded retry */ } throw e; }

Prevention

Set adequate max_tokens and instruct 'raw JSON only, no markdown fences'
Retry once — malformed JSON from LLMs is often stochastic
Log raw model text on parse failure to spot provider error pages

When it happens

Trigger: generate() receives a model reply like prose ("Here are the values: ..."), markdown-fenced JSON handled incorrectly, truncated JSON cut off by max_tokens, or an error page/HTML from the provider instead of JSON.

Common situations: Model wraps JSON in ```json fences or adds commentary; max_tokens too small so the JSON is cut mid-string; provider returning an HTML error page that bypasses response checks; weak models ignoring the JSON instruction; temperature too high producing chatty output.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of linshenkx/prompt-optimizer@3e677b1d9f (2026-08-27). Data as JSON: /api/errors/ba47c673205dc54d. Report an issue: GitHub.