ankane/searchkick · error · Searchkick::Error

Scroll id has expired

Error message

Scroll id has expired

What it means

Raised from Results#scroll when Elasticsearch answers the follow-up scroll request with `search_context_missing_exception` (matched at results.rb:196-201). A scroll context lives only as long as its keep-alive window (the `scroll:` value), refreshed on every continuation call. If the next `.scroll` arrives after the window elapsed, or after `clear_scroll` already freed the cursor, the server no longer holds the context and Searchkick surfaces it as this Error.

Source

Thrown at lib/searchkick/results.rb:198

        records = self
        while records.any?
          yield records
          records = records.scroll
        end

        records.clear_scroll
      else
        begin
          # TODO Active Support notifications for this scroll call
          params = {
            scroll: options[:scroll],
            body: {scroll_id: scroll_id}
          }
          Searchkick.add_opaque_id(params, options[:opaque_id]) if options[:opaque_id]
          Results.new(@klass, Searchkick.client.scroll(params), @options)
        rescue => e
          if Searchkick.not_found_error?(e) && e.message =~ /search_context_missing_exception/i
            raise Error, "Scroll id has expired"
          else
            raise e
          end
        end
      end
    end

    def clear_scroll
      begin
        # try to clear scroll
        # not required as scroll will expire
        # but there is a cost to open scrolls
        Searchkick.client.clear_scroll({body: {scroll_id: scroll_id}})
      rescue => e
        raise e unless Searchkick.transport_error?(e)
      end
    end

View on GitHub (pinned to 93e901a75b)

Solutions

1. Size the keep-alive to worst-case batch time, e.g. `scroll: '5m'` or `'10m'` - the window refreshes on every `.scroll` call, so it only needs to cover one batch, not the whole job
2. Issue the next `.scroll` promptly after processing a batch; keep heavy work out of the gap between receiving a batch and requesting the next one
3. Only call `clear_scroll` after iteration finishes - the block form of `.scroll` already does this
4. If the context is already gone, restart from a fresh search with a new scroll cursor and skip already-processed records via a checkpoint; an expired scroll_id cannot be revived

Example fix

# before
batch = Product.search("*", scroll: "1m")
while batch.any?
  slow_export(batch) # takes > 1 minute per batch
  batch = batch.scroll # raises: search context missing
end

# after
batch = Product.search("*", scroll: "10m") # keep-alive >= worst-case batch time
while batch.any?
  process(batch)
  batch = batch.scroll # each call refreshes the 10m window
end
batch.clear_scroll

# Guard the gap between scroll calls against the keep-alive window
last_call = nil
window_secs = 300 # matches scroll: "5m"

scroll_next = lambda do |results|
  now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  if last_call && (now - last_call) > window_secs * 0.8
    raise CursorStale, "restart scan instead of guaranteeing an expired context"
  end
  batch = results.scroll
  last_call = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  batch
end

begin
  batch = batch.scroll
rescue Searchkick::Error => e
  raise unless e.message == "Scroll id has expired"
  batch = Product.search("*", scroll: "10m") # fresh cursor
  # skip past ids already processed from the checkpoint
end

When it happens

Trigger: `Product.search('*', scroll: '1m')` where per-batch processing takes longer than one minute before the next `.scroll` call; calling `clear_scroll` mid-iteration and scrolling again; resuming a scroll_id saved from a previous run after its window passed; debugger stops, GC pauses, or background-job queue delays stretching the gap between batches.

Common situations: Batch exports over large indexes with slow per-record work; copying `scroll: '1m'` from tutorials regardless of batch cost; retry logic that reuses a stale scroll_id after a worker crash; per-batch side effects that hit external APIs or rate limits.

Related errors

• No search context found for id
• Pass `scroll` option to the search method for scrolling
• Unknown model for index: #{index}. Pass the `models` option
• Multiple clients found - set Searchkick.client_type = :elast
• No client found - install the `elasticsearch` or `opensearch

AI-assisted analysis of ankane/searchkick@93e901a75b (2026-08-21). Data as JSON: /api/errors/4abfd363a0f2af35. Report an issue: GitHub.