{"record":{"id":"3d9adcea680df524","repo":"we-promise/sure","slug":"too-many-pages","errorCode":"too_many_pages","errorMessage":"#{operation_name} exceeded #{MAX_PAGES} pages without exhausting results","messagePattern":"#(.+?) exceeded #(.+?) pages without exhausting results","errorType":"exception","errorClass":"Provider::Redbark::Error","httpStatus":null,"severity":"error","filePath":"app/models/provider/redbark.rb","lineNumber":183,"sourceCode":"          return [ results, true ]\n        end\n\n        pagination = page[:pagination] || {}\n        unless pagination[:hasMore]\n          exhausted = true\n          break\n        end\n\n        # hasMore with an empty page means the server stopped early\n        if data.empty?\n          raise Error.new(\"#{operation_name} returned an empty page while reporting more results\", :truncated)\n        end\n\n        offset += data.size\n      end\n\n      unless exhausted\n        raise Error.new(\"#{operation_name} exceeded #{MAX_PAGES} pages without exhausting results\", :too_many_pages)\n      end\n\n      [ results, false ]\n    end\n\n    def with_retries(operation_name, max_retries: MAX_RETRIES)\n      retries = 0\n\n      begin\n        yield\n      rescue *RETRYABLE_ERRORS, RateLimitError, ServerError => e\n        retries += 1\n\n        if retries <= max_retries\n          delay = calculate_retry_delay(retries)\n          Rails.logger.warn(\n            \"Redbark API: #{operation_name} failed (attempt #{retries}/#{max_retries}): \" \\\n            \"#{e.class}: #{e.message}. Retrying in #{delay}s...\"","sourceCodeStart":165,"sourceCodeEnd":201,"githubUrl":"https://github.com/we-promise/sure/blob/e69894adb92547273377398c15f45c979cd9416a/app/models/provider/redbark.rb#L165-L201","documentation":"Raised by Provider::Redbark's paginate helper when it has fetched MAX_PAGES (50) pages and the server still reports pagination.hasMore: true. It is a deliberate safety cap (see the MAX_PAGES comment: 'so a bad hasMore can never loop forever') — 50 pages equals 10,000 accounts or 25,000 transactions per window. Error type is :too_many_pages.","triggerScenarios":"A server bug where hasMore never flips to false; offset pagination drifting because rows are inserted/deleted mid-iteration; a legitimately huge result set exceeding 50 pages within one bounded window.","commonSituations":"New transactions posted while a long pagination loop runs (offset shift); Redbark pagination regression; importing a decade of history in a single wide window.","solutions":["Narrow the date window for get_transactions so each call needs fewer pages","Retry — transient drift during live inserts often resolves on a fresh run","Report to Redbark if hasMore stays true past the real dataset (server-side bug)","Only if the dataset legitimately exceeds 50 pages, raise MAX_PAGES in a patch and re-verify — it exists to stop infinite loops, not to cap real data"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"def window_needs_split?(start_date, end_date, rows_per_page = 500, max_pages = 50)\n  return true if start_date.nil? || end_date.nil?\n  (end_date - start_date).to_i > max_pages * rows_per_page # crude bound: >1 row/day/page\nend","typeGuard":"def redbark_too_many_pages?(error)\n  error.is_a?(Provider::Redbark::Error) && error.error_type == :too_many_pages\nend","tryCatchPattern":"begin\n  redbark.get_transactions(connection_id: cid, account_id: aid, start_date: from, end_date: to)\nrescue Provider::Redbark::Error => e\n  raise unless e.error_type == :too_many_pages\n  split_and_retry(cid, aid, from, to) # halve the window and enqueue both halves\nend","preventionTips":["Size transaction windows so expected rows stay well under 25k (50 pages x 500)","Treat :too_many_pages like truncation: split the window rather than widening retries","Watch for it recurring on one account — that indicates a stuck hasMore server bug worth reporting"],"tags":["redbark","pagination","loop-guard","has-more","limit-exceeded"],"backgroundTag":"pagination-infinite-loop-cap","analyzedSha":"e69894adb92547273377398c15f45c979cd9416a","analyzedAt":"2026-08-21T18:22:41.165Z","schemaVersion":2},"datasetVersion":"2026-08-21T23:17:16.201Z"}