{"record":{"id":"4abfd363a0f2af35","repo":"ankane/searchkick","slug":"scroll-id-has-expired","errorCode":null,"errorMessage":"Scroll id has expired","messagePattern":"Scroll id has expired","errorType":"exception","errorClass":"Searchkick::Error","httpStatus":null,"severity":"error","filePath":"lib/searchkick/results.rb","lineNumber":198,"sourceCode":"        records = self\n        while records.any?\n          yield records\n          records = records.scroll\n        end\n\n        records.clear_scroll\n      else\n        begin\n          # TODO Active Support notifications for this scroll call\n          params = {\n            scroll: options[:scroll],\n            body: {scroll_id: scroll_id}\n          }\n          Searchkick.add_opaque_id(params, options[:opaque_id]) if options[:opaque_id]\n          Results.new(@klass, Searchkick.client.scroll(params), @options)\n        rescue => e\n          if Searchkick.not_found_error?(e) && e.message =~ /search_context_missing_exception/i\n            raise Error, \"Scroll id has expired\"\n          else\n            raise e\n          end\n        end\n      end\n    end\n\n    def clear_scroll\n      begin\n        # try to clear scroll\n        # not required as scroll will expire\n        # but there is a cost to open scrolls\n        Searchkick.client.clear_scroll({body: {scroll_id: scroll_id}})\n      rescue => e\n        raise e unless Searchkick.transport_error?(e)\n      end\n    end\n","sourceCodeStart":180,"sourceCodeEnd":216,"githubUrl":"https://github.com/ankane/searchkick/blob/93e901a75b11a25101668a616e006b158251b16e/lib/searchkick/results.rb#L180-L216","documentation":"Raised from Results#scroll when Elasticsearch answers the follow-up scroll request with `search_context_missing_exception` (matched at results.rb:196-201). A scroll context lives only as long as its keep-alive window (the `scroll:` value), refreshed on every continuation call. If the next `.scroll` arrives after the window elapsed, or after `clear_scroll` already freed the cursor, the server no longer holds the context and Searchkick surfaces it as this Error.","triggerScenarios":"`Product.search('*', scroll: '1m')` where per-batch processing takes longer than one minute before the next `.scroll` call; calling `clear_scroll` mid-iteration and scrolling again; resuming a scroll_id saved from a previous run after its window passed; debugger stops, GC pauses, or background-job queue delays stretching the gap between batches.","commonSituations":"Batch exports over large indexes with slow per-record work; copying `scroll: '1m'` from tutorials regardless of batch cost; retry logic that reuses a stale scroll_id after a worker crash; per-batch side effects that hit external APIs or rate limits.","solutions":["Size the keep-alive to worst-case batch time, e.g. `scroll: '5m'` or `'10m'` - the window refreshes on every `.scroll` call, so it only needs to cover one batch, not the whole job","Issue the next `.scroll` promptly after processing a batch; keep heavy work out of the gap between receiving a batch and requesting the next one","Only call `clear_scroll` after iteration finishes - the block form of `.scroll` already does this","If the context is already gone, restart from a fresh search with a new scroll cursor and skip already-processed records via a checkpoint; an expired scroll_id cannot be revived"],"exampleFix":"# before\nbatch = Product.search(\"*\", scroll: \"1m\")\nwhile batch.any?\n  slow_export(batch) # takes > 1 minute per batch\n  batch = batch.scroll # raises: search context missing\nend\n\n# after\nbatch = Product.search(\"*\", scroll: \"10m\") # keep-alive >= worst-case batch time\nwhile batch.any?\n  process(batch)\n  batch = batch.scroll # each call refreshes the 10m window\nend\nbatch.clear_scroll","handlingStrategy":"retry","validationCode":"# Guard the gap between scroll calls against the keep-alive window\nlast_call = nil\nwindow_secs = 300 # matches scroll: \"5m\"\n\nscroll_next = lambda do |results|\n  now = Process.clock_gettime(Process::CLOCK_MONOTONIC)\n  if last_call && (now - last_call) > window_secs * 0.8\n    raise CursorStale, \"restart scan instead of guaranteeing an expired context\"\n  end\n  batch = results.scroll\n  last_call = Process.clock_gettime(Process::CLOCK_MONOTONIC)\n  batch\nend","typeGuard":null,"tryCatchPattern":"begin\n  batch = batch.scroll\nrescue Searchkick::Error => e\n  raise unless e.message == \"Scroll id has expired\"\n  batch = Product.search(\"*\", scroll: \"10m\") # fresh cursor\n  # skip past ids already processed from the checkpoint\nend","preventionTips":["Pick a keep-alive at least 2-3x the slowest batch's processing time","Persist the last processed id/offset so a dead cursor is restartable rather than fatal","Never call clear_scroll before iteration completes","Treat scroll cursors as single-run resources, not resumable tokens"],"tags":["searchkick","elasticsearch","scroll","timeout","expired-context","ruby"],"backgroundTag":"cursor-expired","analyzedSha":"93e901a75b11a25101668a616e006b158251b16e","analyzedAt":"2026-08-21T19:06:23.767Z","schemaVersion":2},"datasetVersion":"2026-08-21T23:17:16.201Z"}