we-promise/sure · error · Provider::Redbark::Error

too_many_pages

too_many_pages

Error message

#{operation_name} exceeded #{MAX_PAGES} pages without exhausting results

What it means

Raised by Provider::Redbark's paginate helper when it has fetched MAX_PAGES (50) pages and the server still reports pagination.hasMore: true. It is a deliberate safety cap (see the MAX_PAGES comment: 'so a bad hasMore can never loop forever') — 50 pages equals 10,000 accounts or 25,000 transactions per window. Error type is :too_many_pages.

Source

Thrown at app/models/provider/redbark.rb:183

          return [ results, true ]
        end

        pagination = page[:pagination] || {}
        unless pagination[:hasMore]
          exhausted = true
          break
        end

        # hasMore with an empty page means the server stopped early
        if data.empty?
          raise Error.new("#{operation_name} returned an empty page while reporting more results", :truncated)
        end

        offset += data.size
      end

      unless exhausted
        raise Error.new("#{operation_name} exceeded #{MAX_PAGES} pages without exhausting results", :too_many_pages)
      end

      [ results, false ]
    end

    def with_retries(operation_name, max_retries: MAX_RETRIES)
      retries = 0

      begin
        yield
      rescue *RETRYABLE_ERRORS, RateLimitError, ServerError => e
        retries += 1

        if retries <= max_retries
          delay = calculate_retry_delay(retries)
          Rails.logger.warn(
            "Redbark API: #{operation_name} failed (attempt #{retries}/#{max_retries}): " \
            "#{e.class}: #{e.message}. Retrying in #{delay}s..."

View on GitHub (pinned to e69894adb9)

Solutions

Narrow the date window for get_transactions so each call needs fewer pages
Retry — transient drift during live inserts often resolves on a fresh run
Report to Redbark if hasMore stays true past the real dataset (server-side bug)
Only if the dataset legitimately exceeds 50 pages, raise MAX_PAGES in a patch and re-verify — it exists to stop infinite loops, not to cap real data

Defensive patterns

Strategy: try-catch

Validate before calling

def window_needs_split?(start_date, end_date, rows_per_page = 500, max_pages = 50)
  return true if start_date.nil? || end_date.nil?
  (end_date - start_date).to_i > max_pages * rows_per_page # crude bound: >1 row/day/page
end

Type guard

def redbark_too_many_pages?(error)
  error.is_a?(Provider::Redbark::Error) && error.error_type == :too_many_pages
end

Try / catch

begin
  redbark.get_transactions(connection_id: cid, account_id: aid, start_date: from, end_date: to)
rescue Provider::Redbark::Error => e
  raise unless e.error_type == :too_many_pages
  split_and_retry(cid, aid, from, to) # halve the window and enqueue both halves
end

Prevention

Size transaction windows so expected rows stay well under 25k (50 pages x 500)
Treat :too_many_pages like truncation: split the window rather than widening retries
Watch for it recurring on one account — that indicates a stuck hasMore server bug worth reporting

When it happens

Trigger: A server bug where hasMore never flips to false; offset pagination drifting because rows are inserted/deleted mid-iteration; a legitimately huge result set exceeding 50 pages within one bounded window.

Common situations: New transactions posted while a long pagination loop runs (offset shift); Redbark pagination regression; importing a decade of history in a single wide window.

Related errors

AI-assisted analysis of we-promise/sure@e69894adb9 (2026-08-21). Data as JSON: /api/errors/3d9adcea680df524. Report an issue: GitHub.