we-promise/sure · error · Provider::Openai::Error

Could not parse JSON from response: #{raw.truncate(200)}

Error message

Could not parse JSON from response: #{raw.truncate(200)}

What it means

Raised by Provider::Openai::AutoCategorizer#parse_json_flexibly after all four recovery strategies fail: direct JSON.parse, stripping markdown code fences, unwrapping concatenated/fragments, and a last-resort regex grab of {...}. It includes the first 200 chars of the raw text, which is the key diagnostic — truncated JSON from token limits and pure-prose answers are the two dominant causes. strip_thinking_tags already handles <think>...</think> blocks before this point, so unclosed thinking from a cut-off reasoning model also lands here.

Source

Thrown at app/models/provider/openai/auto_categorizer.rb:436

        end
        # Try greedy match if non-greedy failed
        begin
          return JSON.parse($1)
        rescue JSON::ParserError
          # Continue to next strategy
        end
      end

      # Strategy 4: Find any JSON object (last resort)
      if cleaned =~ /(\{[\s\S]*\})/m
        begin
          return JSON.parse($1)
        rescue JSON::ParserError
          # Fall through to error
        end
      end

      raise Provider::Openai::Error, "Could not parse JSON from response: #{raw.truncate(200)}"
    end

    # Strip thinking model tags (<think>...</think>) from response
    # Some models like Qwen-thinking output reasoning in these tags before the actual response
    def strip_thinking_tags(raw)
      # Remove <think>...</think> blocks but keep content after them
      # If no closing tag, the model may have been cut off - try to extract JSON from inside
      if raw.include?("<think>")
        # Check if there's content after the thinking block
        if raw =~ /<\/think>\s*([\s\S]*)/m
          after_thinking = $1.strip
          return after_thinking if after_thinking.present?
        end
        # If no content after </think> or no closing tag, look inside the thinking block
        # The JSON might be the last thing in the thinking block
        if raw =~ /<think>([\s\S]*)/m
          return $1
        end

View on GitHub (pinned to e69894adb9)

Solutions

1. Read the truncated raw in the message: an abrupt cut (no closing brace) means raise the token budget or lower LLM_MAX_ITEMS_PER_CALL; prose means strengthen the prompt.
2. Add json_object response_format (or strict schema) to the request so the model cannot answer in prose.
3. Retry once with a smaller batch — halving items per call resolves most truncation cases.
4. If using a thinking model via custom provider, ensure the prompt forces the answer after </think> and the budget covers the reasoning.

Example fix

# before
raise Provider::Openai::Error, "Could not parse JSON from response: #{raw.truncate(200)}"

# after
if raw.truncated_json?
  retry_with_smaller_batch # halve batch size and call again
else
  raise Provider::Openai::Error, "Could not parse JSON from response: #{raw.truncate(200)}"
end

begin
  parsed = parse_json_flexibly(raw)
rescue Provider::Openai::Error
  parsed = retry_with(smaller_batch: true) # truncation is the usual cause; halve and re-ask
end

When it happens

Trigger: Response truncated mid-JSON because max_tokens was hit (unterminated strings/braces); model answers in prose with no JSON at all; reasoning model's <think> block never closed because generation was cut; JSON containing an embedded regex-breaking brace imbalance (rare).

Common situations: Large transaction batches against small token budgets; gpt-4.1-mini or third-party models with weaker JSON discipline; Setting.llm_max_items_per_call raised without raising the response budget; custom gateways with lower default max_tokens than OpenAI.

Related errors

• Could not parse JSON from PDF processing response: #{raw.tru
• Invalid JSON in native categorization: #{e.message}
• Could not find categorizations in response
• {e.record.errors.full_messages.to_sentence.presence || e.mes
• OpenAI stream ended without a completion event. This usually

AI-assisted analysis of we-promise/sure@e69894adb9 (2026-08-21). Data as JSON: /api/errors/4b2aefb32b2b12dc. Report an issue: GitHub.