{"record":{"id":"f8759bd6e9862d77","repo":"instructure/canvas-lms","slug":"quiz-response-missing-items","errorCode":null,"errorMessage":"Quiz response missing items","messagePattern":"Quiz response missing items","errorType":"exception","errorClass":"CedarUnavailable","httpStatus":null,"severity":"error","filePath":"app/services/study_assist.rb","lineNumber":290,"sourceCode":"\n    def build_response(tool_key, raw)\n      case tool_key\n      when :summarize then build_summarize_response(raw)\n      when :quiz then build_quiz_response(raw)\n      when :flashcards then build_flashcards_response(raw)\n      end\n    end\n\n    def build_summarize_response(raw)\n      text = raw.to_s.strip\n      raise CedarUnavailable, \"Summary response missing\" if text.blank?\n\n      { response: text }\n    end\n\n    def build_quiz_response(raw)\n      items = parse_json_array!(raw)\n      raise CedarUnavailable, \"Quiz response missing items\" if items.blank? || !items.is_a?(Array)\n\n      quiz_items = items.first(10).map do |item|\n        question = item[:question] || item[\"question\"]\n        options = item[:options] || item[\"options\"]\n        result = item[:result] || item[\"result\"]\n        raise CedarUnavailable, \"Quiz item malformed\" if question.blank? || options.blank? || result.nil?\n\n        { question:, answers: options, correctAnswerIndex: result.to_i }\n      end\n\n      { quizItems: quiz_items }\n    end\n\n    def build_flashcards_response(raw)\n      cards = parse_json_array!(raw)\n      raise CedarUnavailable, \"Flashcards response missing items\" if cards.blank? || !cards.is_a?(Array)\n\n      flash_cards = cards.first(10).map do |card|","sourceCodeStart":272,"sourceCodeEnd":308,"githubUrl":"https://github.com/instructure/canvas-lms/blob/1c9f0bb8013ed69c4f2efe11fd483025469b7e6c/app/services/study_assist.rb#L272-L308","documentation":"build_quiz_response parses the Cedar completion as a JSON array via parse_json_array! and raises CedarUnavailable when the parsed array is blank or not an Array. The model was expected to emit an array of quiz items but did not.","triggerScenarios":"build_response -> build_quiz_response where parse_json_array! returned nil->handled earlier, an empty array, or a non-array (e.g. a JSON object or scalar) from the quiz prompt.","commonSituations":"The LLM answered in prose instead of JSON; the source text was too short to generate quiz items; the model returned an empty array because nothing in the content was quiz-worthy.","solutions":["Retry the quiz generation — non-deterministic LLM output often succeeds on a second call.","Provide richer source content; very small inputs frequently yield zero items.","Log raw Cedar output when this occurs and, if it recurs, tighten the quiz prompt to mandate a JSON array.","Check InstLLMHelper.extract_json_array behavior against the model's output format."],"exampleFix":"// before\n# fragile expectation that model always returns an array\n\n// after\nbegin\n  quiz = service.call(tool: :quiz)\nrescue StudyAssist::CedarUnavailable\n  quiz = service.call(tool: :quiz) # one retry for transient empty output\nend","handlingStrategy":"retry","validationCode":null,"typeGuard":"def usable_quiz_array?(raw)\n  parsed = InstLLMHelper.extract_json_array(raw.to_s)\n  parsed.is_a?(Array) && parsed.any?\nend","tryCatchPattern":"begin\n  quiz = service.call(tool: :quiz)\nrescue StudyAssist::CedarUnavailable\n  quiz = service.call(tool: :quiz) # one retry for transient empty output\nend","preventionTips":["Supply content rich enough to generate quiz items","Keep the quiz prompt strict about returning a JSON array","Log raw completions to detect recurring empty-output patterns"],"tags":["llm","quiz","empty-response","json"],"backgroundTag":"empty-result-set","analyzedSha":"1c9f0bb8013ed69c4f2efe11fd483025469b7e6c","analyzedAt":"2026-09-15T20:33:18.891Z","contentChangedAt":"2026-09-15T20:33:18.891Z","schemaVersion":2},"datasetVersion":"2026-09-23T02:17:17.105Z"}