ankane/searchkick · error · Searchkick::ImportError

#{first_with_error["error"]} on item with id '#{first_with_e

Error message

#{first_with_error["error"]} on item with id '#{first_with_error["_id"]}'

What it means

The `Searchkick::Indexer` batches records into a single Elasticsearch/OpenSearch `_bulk` request. If the response reports `errors: true`, searchkick walks the per-item results, skips deletes of missing documents and items flagged `ignore_missing`, and raises `Searchkick::ImportError` with the first item's raw error and its `_id`. So this error wraps the server's own per-document failure — the message before ' on item with id' is the authoritative cause (e.g. mapper parse exceptions, version conflicts, bulk rejections).

Source

Thrown at lib/searchkick/indexer.rb:29

    def queue(items)
      @queued_items.concat(items)
      perform unless Searchkick.callbacks_value == :bulk
    end

    def perform
      items = @queued_items
      @queued_items = []

      return if items.empty?

      response = Searchkick.client.bulk(body: Searchkick.opensearch4_client? ? items.as_json : items)
      if response["errors"]
        # note: delete does not set error when item not found
        first_with_error = response["items"].map do |item|
          (item["index"] || item["delete"] || item["update"])
        end.find.with_index { |item, i| item["error"] && !ignore_missing?(items[i], item["error"]) }
        if first_with_error
          raise ImportError, "#{first_with_error["error"]} on item with id '#{first_with_error["_id"]}'"
        end
      end

      # maybe return response in future
      nil
    end

    private

    def ignore_missing?(item, error)
      error["type"] == "document_missing_exception" && item.instance_variable_defined?(:@ignore_missing)
    end
  end
end

View on GitHub (pinned to 93e901a75b)

Solutions

1. Read the error text before ' on item with id' — it names the document and the server-side reason (e.g. 'mapper [price] cannot be changed from [long] to [text]', 'version_conflict_engine_exception', 'es_rejected_execution').
2. Load the failing record by the reported id and fix its `search_data` output so it matches the current mapping, then re-run the import.
3. If the error is a mapping/parser conflict after changing `searchkick` options, run a full `Model.reindex` with the new options so the index is recreated before records import.
4. For update/destroy callbacks racing a direct-DB delete, pass `ignore_missing: true` to `reindex` or rescue `Searchkick::ImportError` and treat document_missing as benign.
5. For 429/bulk rejections, lower `batch_size` (e.g. `searchkick batch_size: 100`) or use `mode: :async`/`:queue` so retries happen in background jobs.

Example fix

# before
product.update!(price: "twelve") # reindex callback fires bulk
# => Searchkick::ImportError: mapper [price] failed to parse field [...] on item with id '42'

# after
# fix the data / search_data to match the mapping
product.update!(price: 12.0)

# benign delete races:
begin
  product.reindex(ignore_missing: true)
rescue Searchkick::ImportError => e
  raise unless e.message.include?("document_missing_exception")
end

begin
  Product.reindex(batch_size: 500)
rescue Searchkick::ImportError => e
  id = e.message[/on item with id '(.+))'/, 1] || e.message[/on item with id '(.+)'/, 1]
  reason = e.message.split(" on item with id").first
  Rails.logger.error("reindex failed first on id=#{id}: #{reason}")
  if reason.include?("document_missing_exception")
    retry # record already gone; benign
  else
    raise # real data/mapping problem — fix search_data for that record
  end
end

When it happens

Trigger: `Model.reindex`, `model.reindex`, `Searchkick::Index.new(...).import` or any callback-triggered bulk (create/save/destroy with inline/async callbacks) where at least one item fails: a value that doesn't fit the mapping (string into integer field), sending `searchkick_index` data as non-JSON-serializable objects, a stale mapping after changing `searchkick` options without reindexing, or ES bulk queue overflow (429 TOOMANYREQUESTS).

Common situations: Running `reindex` after a model/mapping change while old documents contain incompatible data (dates as strings, nil into keyword, arrays into scalar fields); data serialization bugs in `search_data`; bulk indexing faster than the cluster accepts (thread_pool.search.queue_size / bulk queue rejections); document_missing_exception on update callbacks after a record was deleted directly in the database (use `model.reindex(..., ignore_missing: true)` or `delete` instead).

Related errors

• Multiple clients found - set Searchkick.client_type = :elast
• No client found - install the `elasticsearch` or `opensearch
• Need primary key to load records
• Not sure how to load records
• Could not find class: #{class_name}

AI-assisted analysis of ankane/searchkick@93e901a75b (2026-08-21). Data as JSON: /api/errors/972722af4e46d8c7. Report an issue: GitHub.