{"record":{"id":"9482c1e0a86745fa","repo":"666ghj/MiroFish","slug":"github-graphql-exceeded-the-page-safety-limit","errorCode":null,"errorMessage":"GitHub GraphQL exceeded the page safety limit","messagePattern":"GitHub GraphQL exceeded the page safety limit","errorType":"exception","errorClass":"StarHistoryError","httpStatus":null,"severity":"error","filePath":"scripts/star_history.py","lineNumber":604,"sourceCode":"                raise StarHistoryError(\"insufficient GitHub GraphQL rate limit for backfill\")\n        for edge in page.edges:\n            if edge.cursor in seen_edge_cursors:\n                raise StarHistoryError(\"GitHub GraphQL repeated an edge cursor\")\n            seen_edge_cursors.add(edge.cursor)\n            if edge.starred_at.date() < normalized_now.date():\n                daily_increments[edge.starred_at.date()] += 1\n\n        if page.end_cursor is not None:\n            if page.end_cursor in seen_page_cursors:\n                raise StarHistoryError(\"GitHub GraphQL repeated a page cursor\")\n            seen_page_cursors.add(page.end_cursor)\n        if not page.has_next_page:\n            break\n        if page.end_cursor is None:\n            raise StarHistoryError(\"GitHub GraphQL omitted the next page cursor\")\n        after = page.end_cursor\n        if page_number > 10_000:\n            raise StarHistoryError(\"GitHub GraphQL exceeded the page safety limit\")\n\n    if initial_total is None:\n        raise StarHistoryError(\"GitHub GraphQL returned no pages\")\n    if len(seen_edge_cursors) != initial_total:\n        raise StarHistoryError(\"stargazer list changed or was incomplete during backfill\")\n\n    running = 0\n    daily: list[dict[str, Any]] = []\n    for point_day in sorted(daily_increments):\n        running += daily_increments[point_day]\n        daily.append({\"date\": point_day.isoformat(), \"stars\": running})\n\n    state: dict[str, Any] = {\n        \"schema_version\": 1,\n        \"repository\": REPOSITORY,\n        \"timezone\": \"UTC\",\n        \"ongoing_interval_days\": INTERVAL_DAYS,\n        \"reconstruction\": {","sourceCodeStart":586,"sourceCodeEnd":622,"githubUrl":"https://github.com/666ghj/MiroFish/blob/b5b53acc57189a4a42e44a23e149dc655c98fe82/scripts/star_history.py#L586-L622","documentation":"Raised at scripts/star_history.py:604 after `page_number` exceeds 10,000 in the backfill `while True` loop. With PAGE_SIZE=100 this caps a single backfill at 1,000,000 stargazers. It is a deliberate safety limit so a cursor bug (see the repeated-cursor guards) cannot turn into an unbounded, rate-limit-burning fetch loop.","triggerScenarios":"Calling the stargazer backfill on a repository whose `totalCount` implies more than 10,000 pages (stars > 1,000,000), or a pagination malfunction that keeps yielding new-but-non-terminating pages so `page_number` keeps climbing past 10,000 without `has_next_page` ever becoming false.","commonSituations":"Extremely popular repositories (1M+ stars — beyond every real repo today, so in practice this fires only in tests or when pagination is malfunctioning); synthetic test states with huge `total_count`; a fake gateway that always says has_next_page=True with fresh cursors.","solutions":["Confirm the repository actually has ~1M+ stargazers via `gh api repos/{owner}/{repo}`; if it does, the backfill design does not support it — raise the safety limit in a fork or backfill in chunks by date.","If the star count is far below 1M, the loop is not terminating: check that the gateway's `has_next_page` eventually becomes False and that `end_cursor` advances (see the repeated/omitted-cursor errors).","Re-run during a quiet window so concurrent churn cannot extend pagination indefinitely.","Inspect logs for how many distinct edge cursors were collected versus `initial_total` to distinguish a genuine huge repo from a looping traversal."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"# Before the backfill, check the repo size against the hard cap\nimport subprocess, json\ninfo = json.loads(subprocess.run(\n    [\"gh\", \"api\", \"repos/666ghj/MiroFish\"], capture_output=True, text=True, check=True\n).stdout)\nif info[\"stargazers_count\"] > 10_000 * 100:  # pages safety limit * PAGE_SIZE\n    raise SystemExit(\"repository exceeds backfill page safety limit; backfill unsupported\")","typeGuard":null,"tryCatchPattern":"try:\n    state = run_backfill(gateway, clock)\nexcept StarHistoryError as exc:\n    if \"page safety limit\" in str(exc):\n        raise SystemExit(\n            \"backfill exceeded 10,000 pages (≈1M stars) or pagination never terminated\"\n        ) from exc\n    raise","preventionTips":["Pre-check stargazers_count against 10_000 * PAGE_SIZE before starting the backfill.","If you legitimately need >1M-star support, fork and raise the limit together with a chunked-by-date strategy — do not simply delete the check.","Treat this error as a symptom: verify has_next_page/end_cursor progression first, since real repos are far below the cap.","Budget rate limit for the full run: pages_required = ceil(total/100), and the preflight check enforces rate_remaining covers it plus reserve."],"tags":["github-api","pagination","safety-limit","backfill","infinite-loop-guard"],"backgroundTag":null,"analyzedSha":"b5b53acc57189a4a42e44a23e149dc655c98fe82","analyzedAt":"2026-08-14T22:29:33.146Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}