instructure/canvas-lms · error · CedarAi::Errors::GraderError

Invalid JSON response: could not extract valid JSON array

Error message

Invalid JSON response: could not extract valid JSON array

What it means

safe_parse_json_array attempts extraction, repair, and parse of an LLM/AI grader response expected to be a JSON array. If JSON.parse of the repaired string still raises JSON::ParserError inside the fallback path, it raises CedarAi::Errors::GraderError 'Invalid JSON response: could not extract valid JSON array'.

Solutions

1. Tighten the prompt to require a bare JSON array with no surrounding text or code fences.
2. Validate/repair upstream (strip markdown fences, remove trailing commas) before calling safe_parse_json_array.
3. Retry the model call, optionally with temperature 0 or a stricter response_format.
4. Rescue CedarAi::Errors::GraderError in the grading pipeline and fall back to manual review.

Example fix

// before
result = safe_parse_json_array(raw_model_output)
// after
trimmed = raw_model_output[/\[.*\]/m]&.gsub(/,\s*([\]\]])/, '\1')
result = safe_parse_json_array(trimmed || raw_model_output)

candidate = raw[/\[.*\]/m]
raise CedarAi::Errors::GraderError, 'no array found' unless candidate&.start_with?('[')

def looks_like_json_array?(s)
  s.is_a?(String) && s.strip.start_with?('[') && s.strip.end_with?(']')
end

begin
  grades = safe_parse_json_array(response)
rescue CedarAi::Errors::GraderError => e
  grades = retry_with_stricter_prompt(response)
end

When it happens

Trigger: The inner rescue branch receives model output whose extracted json_like block cannot be parsed even after escape_inner_quotes repair — e.g. trailing commas, unescaped control characters, or a non-array object.

Common situations: LLM returned prose mixed with code fences in an unexpected shape; model changed output format after a prompt or model version bump; response truncated by token limit mid-array.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• AI response appears truncated - the response may have…
• e.message
• Error parsing JSON results from Outcomes Service: #
• Failed to unescape value: #
• filter values must be JSON-compatible types

AI-assisted analysis of instructure/canvas-lms@1c9f0bb801 (2026-09-15). Data as JSON: /api/errors/700d767721b6ed43. Report an issue: GitHub.

Appendix: source

Thrown at app/helpers/json_utils_helper.rb:37

#

module JsonUtilsHelper
  def safe_parse_json_array(response)
    return [] if response.blank?

    begin
      parsed = JSON.parse(response)
      return parsed.is_a?(Array) ? parsed : []
    rescue JSON::ParserError
      if response.include?("[") && response.include?("]")
        json_like = response[response.index("["), response.rindex("]") - response.index("[") + 1]
        repaired = escape_inner_quotes(json_like)

        begin
          parsed = JSON.parse(repaired)
          return parsed.is_a?(Array) ? parsed : []
        rescue JSON::ParserError
          raise CedarAi::Errors::GraderError, "Invalid JSON response: could not extract valid JSON array"
        end
      end
    end

    raise CedarAi::Errors::GraderError, "Invalid JSON response: could not extract valid JSON array"
  end

  def escape_inner_quotes(json_str)
    result = json_str.dup
    key_start_regex = /"([^"\\]*)"\s*:\s*"/

    pos = 0
    while (m = key_start_regex.match(result, pos))
      value_start = m.end(0)
      closing_regex = /"(?=\s*(?:,|\}|\]))/
      closing_match_pos = result.index(closing_regex, value_start)

      break unless closing_match_pos

View on GitHub (pinned to 1c9f0bb801)