{"record":{"id":"99c23f9af9fdb84d","repo":"instructure/canvas-lms","slug":"ai-response-appears-truncated-the-response-may-have-exceeded","errorCode":null,"errorMessage":"AI response appears truncated - the response may have exceeded length limits. Please try with a shorter prompt or fewer criteria.","messagePattern":"AI response appears truncated - the response may have exceeded length limits\\. Please try with a shorter prompt or fewer criteria\\.","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"app/services/rubric_llm_service.rb","lineNumber":794,"sourceCode":"    JSON.pretty_generate(original)\n  end\n\n  # Extract inner text between XML-like tags in an LLM response.\n  #\n  # Example:\n  #   text = \"... <RUBRIC_DATA>hello</RUBRIC_DATA> ...\"\n  #   extract_text_from_response(text, tag: \"RUBRIC_DATA\") # => \"hello\"\n  #\n  # Raises a more specific error if the response appears truncated (opening tag found but no closing tag).\n  def extract_text_from_response(response_text, tag:)\n    return nil if response_text.blank? || tag.blank?\n\n    regex = %r{<#{Regexp.escape(tag)}>(.*?)</#{Regexp.escape(tag)}>}m\n    match = response_text.match(regex)\n\n    if match.nil? && response_text.include?(\"<#{tag}>\")\n      Rails.logger.error(\"Truncated LLM response detected - opening <#{tag}> found but closing </#{tag}> missing\")\n      raise \"AI response appears truncated - the response may have exceeded length limits. Please try with a shorter prompt or fewer criteria.\"\n    end\n\n    match ? match[1].strip : nil\n  end\n\n  # Quote a value as a JSON string without the surrounding quotes escaping issues.\n  #\n  # Example:\n  #   escape_value(%{She said \"hi\"}) # => \"\\\"She said \\\\\\\"hi\\\\\\\"\\\"\"\n  #   (and we later strip the outer quotes when re-parsing)\n  def escape_value(str)\n    return \"\" if str.nil?\n\n    JSON.generate(str.to_s)[1..-2]\n  end\n\n  # Reverse of escape_value – interpret a line value back into plain text.\n  # If JSON parsing fails (e.g., malformed escape sequences from LLM), returns the original string.","sourceCodeStart":776,"sourceCodeEnd":812,"githubUrl":"https://github.com/instructure/canvas-lms/blob/1c9f0bb8013ed69c4f2efe11fd483025469b7e6c/app/services/rubric_llm_service.rb#L776-L812","documentation":"extract_text_from_response extracts content between <tag>...</tag> markers in the LLM response. If an opening <tag> exists but the closing </tag> is missing, the response was cut off (max token/output length exceeded) and the service raises this user-facing truncation error rather than parsing partial criteria.","triggerScenarios":"LLM response ended mid-XML block because max_output_tokens was hit — typically when generating many criteria, long descriptions, or a long user prompt consuming most of the context/output budget.","commonSituations":"User asks for a large number of detailed criteria; model configured with a small token limit; provider silently truncates long completions; network/provider streaming cut short leaving malformed XML.","solutions":["Retry with a shorter prompt or fewer criteria per request (the error message's own guidance).","Increase the LLM max output token limit in the llm_config used by call_llm_with_prefill.","Instruct the model to be more concise (shorter criterion descriptions) to fit within the output limit.","Split generation into multiple smaller requests (fewer criteria each)."],"exampleFix":"// before\ncriteria = service.generate_criteria_via_llm(prompt: huge_prompt, criteria_count: 12)\n// after\n# smaller batch + explicit brevity instruction\nprompt << \"\\nKeep each criterion description under 200 characters.\"\ncriteria = service.generate_criteria_via_llm(prompt: prompt, criteria_count: 4)\nrescue RuntimeError => e\n  raise e unless e.message.include?(\"AI response appears truncated\")\n  retry with fewer criteria / higher token limit","handlingStrategy":"retry","validationCode":"# estimate output size before calling\nreturn :too_large if requested_criteria_count > 6 # heuristic cap to avoid truncation","typeGuard":null,"tryCatchPattern":"begin\n  criteria = service.generate_criteria_via_llm(...)\nrescue RuntimeError => e\n  raise unless e.message.include?('AI response appears truncated')\n  retry_with(fewer_criteria: true, larger_token_limit: true)\nend","preventionTips":["Cap the number of criteria requested per LLM call.","Configure a generous max output token limit in llm_config.","Prompt the model for concise descriptions to stay under output limits.","Monitor logs for the 'Truncated LLM response detected' error line to tune limits."],"tags":["llm","truncated-response","token-limit","parsing"],"backgroundTag":"payload-too-large","analyzedSha":"1c9f0bb8013ed69c4f2efe11fd483025469b7e6c","analyzedAt":"2026-09-15T20:33:18.891Z","contentChangedAt":"2026-09-15T20:33:18.891Z","schemaVersion":2},"datasetVersion":"2026-09-23T02:17:17.105Z"}