instructure/canvas-lms · error · CedarUnavailable

Quiz response missing items

Error message

Quiz response missing items

What it means

build_quiz_response parses the Cedar completion as a JSON array via parse_json_array! and raises CedarUnavailable when the parsed array is blank or not an Array. The model was expected to emit an array of quiz items but did not.

Solutions

1. Retry the quiz generation — non-deterministic LLM output often succeeds on a second call.
2. Provide richer source content; very small inputs frequently yield zero items.
3. Log raw Cedar output when this occurs and, if it recurs, tighten the quiz prompt to mandate a JSON array.
4. Check InstLLMHelper.extract_json_array behavior against the model's output format.

Example fix

// before
# fragile expectation that model always returns an array

// after
begin
  quiz = service.call(tool: :quiz)
rescue StudyAssist::CedarUnavailable
  quiz = service.call(tool: :quiz) # one retry for transient empty output
end

def usable_quiz_array?(raw)
  parsed = InstLLMHelper.extract_json_array(raw.to_s)
  parsed.is_a?(Array) && parsed.any?
end

begin
  quiz = service.call(tool: :quiz)
rescue StudyAssist::CedarUnavailable
  quiz = service.call(tool: :quiz) # one retry for transient empty output
end

When it happens

Trigger: build_response -> build_quiz_response where parse_json_array! returned nil->handled earlier, an empty array, or a non-array (e.g. a JSON object or scalar) from the quiz prompt.

Common situations: The LLM answered in prose instead of JSON; the source text was too short to generate quiz items; the model returned an empty array because nothing in the content was quiz-worthy.

Understand the failure class

Background: EmptyResultError / "no results found": when an API or scraper succeeds but returns zero rows — this error's family across 9 libraries.

Related errors

• Flashcards response missing items
• Invalid JSON response from Cedar
• Quiz item malformed
• Summary response missing
• The AI response was not in the expected format. Please try…

AI-assisted analysis of instructure/canvas-lms@1c9f0bb801 (2026-09-15). Data as JSON: /api/errors/f8759bd6e9862d77. Report an issue: GitHub.

Appendix: source

Thrown at app/services/study_assist.rb:290

    def build_response(tool_key, raw)
      case tool_key
      when :summarize then build_summarize_response(raw)
      when :quiz then build_quiz_response(raw)
      when :flashcards then build_flashcards_response(raw)
      end
    end

    def build_summarize_response(raw)
      text = raw.to_s.strip
      raise CedarUnavailable, "Summary response missing" if text.blank?

      { response: text }
    end

    def build_quiz_response(raw)
      items = parse_json_array!(raw)
      raise CedarUnavailable, "Quiz response missing items" if items.blank? || !items.is_a?(Array)

      quiz_items = items.first(10).map do |item|
        question = item[:question] || item["question"]
        options = item[:options] || item["options"]
        result = item[:result] || item["result"]
        raise CedarUnavailable, "Quiz item malformed" if question.blank? || options.blank? || result.nil?

        { question:, answers: options, correctAnswerIndex: result.to_i }
      end

      { quizItems: quiz_items }
    end

    def build_flashcards_response(raw)
      cards = parse_json_array!(raw)
      raise CedarUnavailable, "Flashcards response missing items" if cards.blank? || !cards.is_a?(Array)

      flash_cards = cards.first(10).map do |card|

View on GitHub (pinned to 1c9f0bb801)