instructure/canvas-lms · error · ArgumentError

Item # has invalid final_label: # . Expected one of: #

Error message

Item #{index} has invalid final_label: #{item["final_label"]}. Expected one of: #{valid_labels.join(", ")}

What it means

DiscussionTopicInsight.validate_llm_response checks each LLM-returned item against required fields and an allowed set of final_label values ('relevant', 'needs_review', 'irrelevant'). If an item's final_label is outside that set, it raises ArgumentError naming the item index, the bad value, and the valid options.

Solutions

  1. Tighten the LLM prompt/schema to enforce the exact enum values.
  2. Normalize the label (downcase/trim) before validation.
  3. Reject or retry the generation when any item has an invalid label.
  4. Add few-shot examples showing the exact final_label values.

Example fix

// before
raise ArgumentError unless valid_labels.include?(item["final_label"])
// after
normalized = item["final_label"].to_s.strip.downcase
raise ArgumentError unless valid_labels.include?(normalized)
Defensive patterns

Strategy: validation

Validate before calling

VALID_LABELS = %w[relevant needs_review irrelevant]
items.each_with_index do |item, i|
  raise "item #{i}: bad final_label" unless VALID_LABELS.include?(item["final_label"].to_s)
end

Try / catch

begin
  insight.validate_llm_response(response)
rescue ArgumentError => e
  retry_generation_with_stricter_prompt(e.message)
end

Prevention

When it happens

Trigger: generate() parses an LLM response whose item at some index has a final_label not in the allowed enum (typos, model hallucination, prompt/schema drift).

Common situations: LLM outputs 'Relevant' (capitalized), 'not_relevant', or free-text labels; prompt changes weaken the output contract.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

AI-assisted analysis of instructure/canvas-lms@1c9f0bb801 (2026-09-15). Data as JSON: /api/errors/a7c348da2326e31d. Report an issue: GitHub.

Appendix: source

Thrown at app/models/discussion_topic_insight.rb:194

    response_ids = response.pluck("id")
    expected_sequence = (0...response.length).to_a.map(&:to_s)

    if response_ids != expected_sequence
      raise ArgumentError, "Response ids [#{response_ids.join(", ")}] are not sequential numbers starting from 0"
    end

    required_fields = %w[final_label feedback]

    response.each_with_index do |item, index|
      missing_fields = required_fields.select { |field| item[field].nil? }

      unless missing_fields.empty?
        raise ArgumentError, "Item #{index} in LLM response is missing required fields: #{missing_fields.join(", ")}"
      end

      valid_labels = %w[relevant needs_review irrelevant]
      unless valid_labels.include?(item["final_label"])
        raise ArgumentError, "Item #{index} has invalid final_label: #{item["final_label"]}. Expected one of: #{valid_labels.join(", ")}"
      end
    end
  end

  def locale
    discussion_topic.course.locale || I18n.default_locale.to_s
  end

  def unprocessed_entries(should_preload: false)
    student_user_ids = discussion_topic.course.enrollments.active
                                       .where(enrollments: { type: "StudentEnrollment" })
                                       .pluck(:user_id).to_set

    entries = discussion_topic.root_discussion_entries.where(user_id: student_user_ids)
    if should_preload
      entries = entries.preload(:discussion_entry_versions, :user, :attachment)
    end
    entries = entries.to_a

View on GitHub (pinned to 1c9f0bb801)