instructure/canvas-lms · error · CedarUnavailable

Quiz item malformed

Error message

Quiz item malformed

What it means

Within build_quiz_response, each of the (up to 10) parsed quiz items must have non-blank question, non-blank options, and a non-nil result. Any item missing a field raises CedarUnavailable 'Quiz item malformed'.

Solutions

1. Retry — malformed items are typical stochastic LLM output and often fixed on regeneration.
2. Log the offending item to see which field is missing; adjust the quiz prompt to demand exact keys question/options/result.
3. If regeneration is too costly, drop malformed items client-side only when a minimum viable count survives.
4. Pin/verify the Cedar model version if the schema suddenly changed.

Example fix

// before
# hard failure on first malformed item
items.map { |i| build_item!(i) }

// after
valid = items.filter_map { |i| build_item!(i) rescue nil }
raise StudyAssist::CedarUnavailable, 'Quiz response missing items' if valid.empty?

QUIZ_ITEM_SCHEMA = ->(i) {
  q = i[:question] || i['question']
  o = i[:options] || i['options']
  r = i[:result] || i['result']
  q.present? && o.present? && !r.nil?
}
pre_ok = parsed_items.all?(&QUIZ_ITEM_SCHEMA)

def valid_quiz_item?(item)
  q = item[:question] || item['question']
  o = item[:options] || item['options']
  r = item[:result] || item['result']
  q.is_a?(String) && q.present? && o.is_a?(Array) && o.any? && !r.nil?
end

begin
  quiz = service.call(tool: :quiz)
rescue StudyAssist::CedarUnavailable
  Rails.logger.warn('quiz items malformed; regenerating')
  quiz = service.call(tool: :quiz)
end

When it happens

Trigger: build_response -> build_quiz_response when any item in the Cedar JSON array lacks :question/:"question", :options/:"options", or :result/:"result", or has blank question/options / nil result.

Common situations: The LLM emits a mix of valid and malformed items (skipped options, missing answer key); prompt drift after a model upgrade changes the item schema; symbol-vs-string keys handled but values still missing.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Flashcard item malformed
• Invalid fields for a group
• Invalid fields for an outcome
• Invalid JSON response from Cedar
• Item # has invalid final_label: # . Expected one of: #

AI-assisted analysis of instructure/canvas-lms@1c9f0bb801 (2026-09-15). Data as JSON: /api/errors/f2281f2f0e2ebf57. Report an issue: GitHub.

Appendix: source

Thrown at app/services/study_assist.rb:296

      end
    end

    def build_summarize_response(raw)
      text = raw.to_s.strip
      raise CedarUnavailable, "Summary response missing" if text.blank?

      { response: text }
    end

    def build_quiz_response(raw)
      items = parse_json_array!(raw)
      raise CedarUnavailable, "Quiz response missing items" if items.blank? || !items.is_a?(Array)

      quiz_items = items.first(10).map do |item|
        question = item[:question] || item["question"]
        options = item[:options] || item["options"]
        result = item[:result] || item["result"]
        raise CedarUnavailable, "Quiz item malformed" if question.blank? || options.blank? || result.nil?

        { question:, answers: options, correctAnswerIndex: result.to_i }
      end

      { quizItems: quiz_items }
    end

    def build_flashcards_response(raw)
      cards = parse_json_array!(raw)
      raise CedarUnavailable, "Flashcards response missing items" if cards.blank? || !cards.is_a?(Array)

      flash_cards = cards.first(10).map do |card|
        question = card[:question] || card["question"]
        answer = card[:answer] || card["answer"]
        raise CedarUnavailable, "Flashcard item malformed" if question.blank? || answer.blank?

        { question:, answer: }
      end

View on GitHub (pinned to 1c9f0bb801)