{"record":{"id":"f707decb31f38a6c","repo":"instructure/canvas-lms","slug":"the-ai-response-was-not-in-the-expected-format-please-try","errorCode":null,"errorMessage":"The AI response was not in the expected format. Please try again.","messagePattern":"The AI response was not in the expected format\\. Please try again\\.","errorType":"exception","errorClass":"JSON::ParserError","httpStatus":null,"severity":"error","filePath":"app/services/rubric_llm_service.rb","lineNumber":272,"sourceCode":"    }\n  end\n\n  def parse_and_transform_generated_criteria(response, generate_options)\n    json_str = \"{\" + response\n    last_index = json_str.rindex(\"}\")\n    json_str = json_str[0..last_index] unless last_index.nil?\n    ai_rubric = JSON.parse(json_str, symbolize_names: true)\n\n    criteria_count = ai_rubric[:criteria].length\n    total_points = generate_options[:total_points].to_f\n    points_per_criterion = calculate_points_per_criterion(total_points, criteria_count)\n\n    ai_rubric[:criteria].each_with_index.map do |criterion_data, index|\n      build_criterion_from_llm(criterion_data, points_per_criterion[index], generate_options[:use_range])\n    end\n  rescue JSON::ParserError => e\n    Rails.logger.error(\"Failed to parse LLM response as JSON during generation: #{e.message}\")\n    raise JSON::ParserError, \"The AI response was not in the expected format. Please try again.\"\n  end\n\n  # Calculate points per criterion based on total_points and criteria_count\n  def calculate_points_per_criterion(total_points, criteria_count)\n    points_per_criterion = (total_points / criteria_count).round(ROUNDING_PRECISION)\n    points_for_criterion = {}\n    running_total = 0.0\n\n    Array(1..criteria_count).each_with_index do |_, index|\n      if index == criteria_count - 1\n        points_for_criterion[index] = (total_points - running_total).round(ROUNDING_PRECISION)\n      else\n        running_total += points_per_criterion\n        points_for_criterion[index] = points_per_criterion\n      end\n    end\n    points_for_criterion\n  end","sourceCodeStart":254,"sourceCodeEnd":290,"githubUrl":"https://github.com/instructure/canvas-lms/blob/1c9f0bb8013ed69c4f2efe11fd483025469b7e6c/app/services/rubric_llm_service.rb#L254-L290","documentation":"JSON::ParserError re-raised by RubricLlmService#parse_and_transform_generated_criteria when the LLM's response for generated rubric criteria cannot be parsed as JSON. The service logs the raw parse failure and re-raises with a user-friendly message asking the caller to retry.","triggerScenarios":"Calling generate_criteria_via_llm when the model returns non-JSON output: markdown-fenced text, prose preamble, truncated response, or an empty/error completion.","commonSituations":"LLM provider returning an error message instead of generated content; max_tokens truncating the JSON; prompt changes causing the model to add explanation text around the JSON; rate-limit fallback bodies.","solutions":["Retry the generation request — the message is explicitly designed to be retried","Log/inspect the raw LLM response to see what invalid output was returned","Strip markdown fences / leading text before JSON.parse, or ask the model for strict JSON-only output","Reduce criteria_count or total_points so the response is not truncated by token limits"],"exampleFix":"// before\nraw = llm_client.complete(prompt)\ncriteria = JSON.parse(raw)\n// after\nraw = llm_client.complete(prompt)\njson = raw.sub(/\\A```(?:json)?\\s*/m, '').sub(/\\s*```\\z/m, '')\ncriteria = JSON.parse(json)","handlingStrategy":"retry","validationCode":"null","typeGuard":"null","tryCatchPattern":"begin\n  criteria = service.generate_criteria_via_llm(...)\nrescue JSON::ParserError\n  # retry up to N times with backoff; surface 'please try again' to the user\n  retry_count += 1\n  retry if retry_count < 3\n  raise\nend","preventionTips":["Request strict JSON-only output (or provider JSON mode) in the LLM call","Log raw responses for diagnosis","Leave headroom on max_tokens so JSON is never truncated","Validate LLM output shape immediately after parsing"],"tags":["ruby","rails","llm","json","rubric","ai"],"backgroundTag":"json-parse-error","analyzedSha":"1c9f0bb8013ed69c4f2efe11fd483025469b7e6c","analyzedAt":"2026-09-15T20:33:18.891Z","contentChangedAt":"2026-09-15T20:33:18.891Z","schemaVersion":2},"datasetVersion":"2026-09-23T02:17:17.105Z"}