linshenkx/prompt-optimizer · error · VariableExtractionParseError

Failed to parse LLM response: ${error instanceof Error ? err

Error message

Failed to parse LLM response: ${error instanceof Error ? error.message : String(error)}. Raw content length: ${content.length} characters.

What it means

The LLM returned a response, but it could not be parsed into the expected JSON extraction result even after repair attempts — the direct JSON.parse fallback failed too. The message includes the original parser error and the raw content length to help diagnose whether the model returned truncated JSON, prose, or a refusal message.

Source

Thrown at packages/core/src/services/variable-extraction/service.ts:182

    try {
      // 2. 使用 jsonrepair 修复可能的格式问题
      const repaired = jsonrepair(jsonText);
      const parsed = JSON.parse(repaired);

      // 3. 标准化响应
      return this.normalizeExtractionResponse(parsed);
    } catch (error) {
      console.warn(
        '[VariableExtractionService] Failed to parse JSON:',
        error instanceof Error ? error.message : String(error)
      );

      // 尝试直接解析（不通过 jsonrepair）
      try {
        const parsed = JSON.parse(jsonText);
        return this.normalizeExtractionResponse(parsed);
      } catch (fallbackError) {
        throw new VariableExtractionParseError(
          `Failed to parse LLM response: ${error instanceof Error ? error.message : String(error)}. Raw content length: ${content.length} characters.`
        );
      }
    }
  }

  /**
   * 标准化提取响应（统一结构）
   */
  private normalizeExtractionResponse(data: any): VariableExtractionResponse {
    if (!data || typeof data !== 'object') {
      throw new VariableExtractionParseError('Extraction result is not a valid object.');
    }

    // 验证 variables 字段
    if (!Array.isArray(data.variables)) {
      throw new VariableExtractionParseError('Extraction result must have a "variables" array.');
    }

View on GitHub (pinned to 3e677b1d9f)

Solutions

Increase max_tokens / output limit so the JSON is not truncated (check 'Raw content length' in the message).
Retry with a stronger model or lower temperature for extraction tasks.
Log the raw LLM content to see exactly what came back (markdown fences, prose, refusal) and adjust the template's JSON instruction accordingly.
Enable or improve JSON repair/preprocessing (strip code fences, extract the first {...} block) before JSON.parse.

Example fix

// before
const parsed = JSON.parse(jsonText);

// after
const m = raw.match(/\{[\s\S]*\}/);
const parsed = JSON.parse(m ? m[0] : jsonText);

Defensive patterns

Strategy: retry

Validate before calling

const looksLikeJson = (s: string) => /[\[{]/.test(s);
// cannot fully validate before the LLM call; use retry with a stricter prompt on failure

Type guard

const isParsableJson = (s: string): boolean => { try { JSON.parse(s); return true; } catch { return false; } };

Try / catch

for (let attempt = 0; attempt < 3; attempt++) {
  try {
    return await extractionService.extract({ modelKey, content });
  } catch (e) {
    if (e instanceof VariableExtractionParseError && attempt < 2) continue; // retry
    throw e;
  }
}

Prevention

Set max_tokens generously so JSON output is never truncated.
Instruct the model to return raw JSON only (no markdown fences, no prose) in the prompt.
Preprocess LLM output: strip code fences and extract the first balanced {...} block before parsing.
Prefer models known to follow JSON output instructions for extraction workloads.

When it happens

Trigger: Calling extract() where the model output is not valid JSON: model wraps JSON in markdown fences or prose (when the stripping logic doesn't catch it), response is truncated by max_tokens, the model returns an empty string or a refusal, or the JSON has trailing commas/quotes that jsonrepair also fails on.

Common situations: Small/cheap models that follow JSON instructions poorly; max_tokens set too low so the JSON is cut off mid-array; prompt template modified so the JSON instruction was weakened; model returning Chinese-language preamble before JSON; temperature too high producing malformed output.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of linshenkx/prompt-optimizer@3e677b1d9f (2026-08-27). Data as JSON: /api/errors/4424cc121a41dfb7. Report an issue: GitHub.