instructure/canvas-lms · warning

Out of region error report received

Error message

Out of region error report received

What it means

ErrorReport creation is refused when the current Shard is not in the current region: writing error reports cross-region would break locality guarantees, so the code sets report = nil and raises 'Out of region error report received' before saving. The rescue logs the failure instead of surfacing it to the user.

Solutions

1. Ensure the job/request runs in the region that owns the shard (correct GuardRail region and job routing)
2. Fix region/shard configuration so Shard.current is in_current_region? for normal traffic
3. For out-of-region errors, log to the local logger/external error service instead of creating an ErrorReport row
4. Verify region setting in config/delayed_jobs and dynamic_settings so processes run in their home region

Example fix

// before
ErrorReport.log_error(:job, e) # runs on out-of-region shard
// after
if Shard.current.in_current_region?
  ErrorReport.log_error(:job, e)
else
  Rails.logger.error("Out of region error: #{e.inspect}")
end

ErrorReport.log_error(:job, e) if Shard.current.in_current_region?

begin
  ErrorReport.log_error(:type, err)
rescue RuntimeError => e
  Rails.logger.error("no ErrorReport created: #{e.message}") # already handled internally, mirror the fallback
end

When it happens

Trigger: log_error/ErrorReport.log_from or Canvas::Errors capture running inside a GuardRail primary block on a shard whose database server lives in a different region than the process (regional failover, cross-region job execution, misrouted request).

Common situations: Multi-region Canvas deployments where a job or request executes against a shard in region B while the app is deployed in region A; region failover tests; misconfigured shard/database region mappings.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• assessor and assessee required
• association required
• can only forward one conversation at a time
• can't accept
• Can't update submission scores unless it's completed

AI-assisted analysis of instructure/canvas-lms@1c9f0bb801 (2026-09-15). Data as JSON: /api/errors/a2386d0b9d605cab. Report an issue: GitHub.

Appendix: source

Thrown at app/models/error_report.rb:105

        new_backtrace = Array(exception.backtrace)
        # remove the common lines of the backtrace, and separate it so you can see
        # the error handling
        backtrace = (new_backtrace - backtrace) + ["<Caused>"] + backtrace
      end
      opts[:message] ||= message
      opts[:backtrace] = backtrace.join("\n")
      opts[:exception_message] = message
      log_error(category, opts)
    end

    def create_error_report(opts)
      GuardRail.activate(:primary) do
        begin
          report = ErrorReport.new
          report.assign_data(opts)
          unless Shard.current.in_current_region?
            report = nil
            raise "Out of region error report received"
          end

          report.save!
          Rails.logger.info("Created ErrorReport ID #{report.global_id}")
        rescue => e
          Rails.logger.error("Failed creating ErrorReport: #{e.inspect}")
          Rails.logger.error("Original error: #{opts[:message]}")
          Rails.logger.error("Original exception: #{opts[:exception_message]}") if opts[:exception_message]
          @exception&.backtrace&.each do |line|
            Rails.logger.error("Trace: #{line}")
          end
        end
        report
      end
    end
  end

  def self.configure_to_ignore(error_classes)

View on GitHub (pinned to 1c9f0bb801)