instructure/canvas-lms · critical · Errors::WrongShard

Operation is being run on the wrong shard. Expected #

Error message

Operation is being run on the wrong shard. Expected #{switchman_shard.id}, got #{Shard.current.id}

What it means

CanvasOperations::Errors::WrongShard raised inside BaseOperation#run when Shard.current does not match the operation's switchman_shard. Because Canvas is sharded via Switchman, running the operation on the wrong shard could corrupt data or run callbacks against the wrong database, so it fails hard instead of using fail_with_error! (failure callbacks may assume the correct shard).

Solutions

Wrap execution in the correct shard: operation.switchman_shard.activate { operation.run } or Shard.wrap(operation.switchman_shard.id) { ... }
Rebuild the operation on the shard you intend to run it on
In delayed jobs, persist and re-activate the shard (e.g. via Switchman's job extension) before run

Example fix

// before
DataFixupOperation.for(account).run
// after
account.shard.activate { DataFixupOperation.for(account).run }

Defensive patterns

Strategy: try-catch

Validate before calling

raise 'wrong shard' unless Shard.current == operation.switchman_shard

Try / catch

rescue CanvasOperations::Errors::WrongShard
  operation.switchman_shard.activate { operation.run }
end

Prevention

Always activate the target shard before run in background jobs and consoles
Persist shard info with the job payload (Switchman job extensions)
Add an assertion in job handlers that Shard.current matches the operation

When it happens

Trigger: Calling operation.run from a context whose current shard differs from the shard the operation was built/assigned to — e.g. enqueuing on the default shard and executing on Shard.default instead of the target account's shard.

Common situations: Background job workers that don't call Shard.wrap/activate_shard before running; creating the operation on a root account but running it under a different shard in a console or scheduled job.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of instructure/canvas-lms@1c9f0bb801 (2026-09-15). Data as JSON: /api/errors/d7eb9bd70c946b3e. Report an issue: GitHub.

Appendix: source

Thrown at lib/canvas_operations/base_operation.rb:131

    #       callbacks (`before_run`, `after_run`, `around_run`).
    # @note When using `run_later`, the progress tracker is managed automatically by `Progress#process_job` unless
    #       progress_tracking is disabled.
    #
    # @note If the operation raises `Errors::InvalidOperationTarget`, the operation gracefully fails, runs any
    #       failure callbacks, and marks the progress as failed.
    #
    # @param new_progress [Progress] The progress tracker object to be used for this operation. Note that
    #   `Progress#process_job` manages this automatically when using `run_later`.
    # @raise [Errors::WrongShard] If the operation is run on an incorrect shard.
    # @return [void]
    def run(new_progress = nil)
      @progress = new_progress if new_progress

      run_callbacks :run do
        unless Shard.current == switchman_shard
          # We intentionally do not call fail_with_error! here because we are on the wrong shard and subclasses may be
          # making assumptions about what shard the failure callbacks run on.
          raise Errors::WrongShard, "Operation is being run on the wrong shard. Expected #{switchman_shard.id}, got #{Shard.current.id}"
        end

        execute

        complete_progress
      end
    rescue Errors::InvalidOperationTarget => e
      log_message("Operation failed due to invalid operation target: #{e.message}", level: :error)
      log_message("Note that the above error is being rescued; if this is a migration, other migrations can still continue.", level: :info)

      results[:error] = e.message

      fail_with_error!
    end

    # Marks the operation as failed and updates the progress with the current results.
    #
    # When `#run_later` is used, the Progress lifecycle is managed automatically, and this method

View on GitHub (pinned to 1c9f0bb801)