instructure/canvas-lms · error · CedarAi::Errors::GraderError

Grading could not be completed. Please try again.

Error message

Grading could not be completed. Please try again.

What it means

AutoGradeOrchestrationService#get_grade_data (invoked by run_auto_grader) merges the AI grader's returned grade_data with existing grades, then validates that every rubric criterion has a grade. If get_criteria_missing_grades is non-empty (criteria count mismatch between merged data and rubric.data), it raises CedarAi::Errors::GraderError with a user-friendly retry message, treating the grader output as incomplete/invalid.

Solutions

1. Retry the auto-grading run — the message is explicitly 'Please try again'; transient grader output issues often resolve on a subsequent attempt.
2. Check the rubric criteria vs the grader output in AutoGradeResult.grade_data/logs and identify which criterion ids are missing.
3. If the rubric was recently edited, re-run grading so prompts are rebuilt from the current rubric.data.
4. Catch CedarAi::Errors::GraderError in the orchestration caller and fall back to manual grading or mark the auto-grade attempt failed.

Example fix

// before
result = AutoGradeOrchestrationService.new(submission:).run_auto_grader # may raise GraderError

// after
begin
  result = AutoGradeOrchestrationService.new(submission:).run_auto_grader
rescue CedarAi::Errors::GraderError => e
  Rails.logger.warn("Auto-grade failed for submission #{submission.id}: #{e.message}")
  AutoGradeOrchestrationService.new(submission:).run_auto_grader # retry once
end

merged = merge_new_grade_data_with_existing(grade_data, existing)
missing = get_criteria_missing_grades(merged, rubric)
raise CedarAi::Errors::GraderError, "missing criteria: #{missing.join(',')}" unless missing.empty?

def complete_grade_data?(grade_data, rubric)
  grade_data.is_a?(Array) && grade_data.length == rubric.data.length
end

begin
  service.run_auto_grader
rescue CedarAi::Errors::GraderError => e
  Rails.logger.warn("Auto-grade incomplete: #{e.message}")
  service.run_auto_grader # retry; grader output is often transiently partial
end

When it happens

Trigger: Run the auto grader (Cedar AI) on a submission whose rubric expects N criteria but the grader's grade_data (after merge) covers fewer criteria or omits some criterion ids — e.g. the model returned a partial or malformed grade payload.

Common situations: LLM/ Cedar grader returning truncated JSON; rubric edited (criteria added/removed) after the grader prompt was built; rubric criteria whose descriptions the grader failed to match; retryable transient model output issues.

Related errors

• [AutoGrade] Criteria count mismatch for submission #
• LLM generated # criteria but expected # . Truncating excess…
• A course did not pass validation
• A # user did not pass validation (user: # , # : # , error…
• A maximum of 50 assessees can be provided at once

AI-assisted analysis of instructure/canvas-lms@1c9f0bb801 (2026-09-15). Data as JSON: /api/errors/80c1837861adcba7. Report an issue: GitHub.

Appendix: source

Thrown at app/services/auto_grade_orchestration_service.rb:98

    missing_criteria = get_criteria_missing_grades(auto_grade_result.grade_data, rubric)

    if missing_criteria.any?
      # filter rubric to only include missing criteria
      relevant_rubric = rubric.data.select { |item| missing_criteria.include?(item[:description]) }

      grade_data = GradeService.new(
        assignment: assignment_text,
        essay: self.class.extract_essay_text(submission),
        rubric: relevant_rubric,
        root_account_uuid:,
        current_user: @current_user
      ).call

      merged_data = merge_new_grade_data_with_existing(grade_data, auto_grade_result.grade_data || [])

      unless get_criteria_missing_grades(merged_data, rubric).empty?
        Rails.logger.warn("[AutoGrade] Criteria count mismatch for submission #{submission.id}: got #{merged_data.length}, expected #{rubric.data.length}")
        raise CedarAi::Errors::GraderError, I18n.t("Grading could not be completed. Please try again.")
      end

      auto_grade_result.update!(
        root_account_id: submission.course.root_account_id,
        grade_data: merged_data,
        error_message: nil,
        grading_attempts: auto_grade_result.grading_attempts + 1
      )
    end

    auto_grade_result if auto_grade_result.persisted?
  rescue => e
    retryable = e.is_a?(CedarAi::Errors::GraderError)
    handle_grading_failure(
      error_message: "Grading failed: #{e.message}",
      submission:,
      auto_grade_result:,
      progress:,

View on GitHub (pinned to 1c9f0bb801)