unslothai/unsloth · error · RuntimeError

REST failed after {max_retries} retries: {last_err}

Error message

REST failed after {max_retries} retries: {last_err}

What it means

RuntimeError raised after the REST request exhausts max_retries attempts (default 6). Retries happen for transport-level requests.RequestException with exponential backoff (capped at 60s), and for 403/429 responses the client sleeps (until rate-limit reset if known, else 60s) before retrying. Only when every attempt fails does this error surface with the last underlying exception.

Source

Thrown at studio/backend/plugins/data-designer-github-repo-seed/src/data_designer_github_repo_seed/scraper_impl/gh_client.py:280

                    retry_after = _retry_after_seconds(r.headers.get("Retry-After"))
                    if retry_after is not None:
                        log.warning("Secondary rate limit on REST. Sleep %ds.", retry_after)
                        time.sleep(retry_after + 2)
                        continue
                    # Primary rate limit
                    if self.rest_remaining == 0 and self.rest_reset:
                        self._sleep_until(self.rest_reset)
                        continue
                    log.warning("REST 403/429, sleep 60")
                    time.sleep(60)
                    continue
                return r
            except requests.RequestException as e:
                last_err = e
                log.warning("REST network error: %s. Retry.", e)
                time.sleep(backoff)
                backoff = min(backoff * 2, 60)
        raise RuntimeError(f"REST failed after {max_retries} retries: {last_err}")

    def rest_paginate(
        self,
        path: str,
        params: Optional[Dict[str, Any]] = None,
        per_page: int = 100,
    ) -> Iterator[dict]:
        params = dict(params or {})
        params.setdefault("per_page", per_page)
        url = path
        while True:
            r = self.rest("GET", url, params = params if url == path else None)
            if r.status_code != 200:
                log.error("REST paginate got %s at %s: %s", r.status_code, url, r.text[:200])
                return
            items = r.json()
            if isinstance(items, dict):
                # Some endpoints wrap the list in an "items" field

View on GitHub (pinned to 203007d190)

Solutions

1. Wait for the rate-limit window to reset (check X-RateLimit-Reset) and re-run; the scraper resumes from cached data.
2. Reduce parallelism or run a single scraper instance per token to stay under primary and secondary rate limits.
3. Fix network/proxy issues if the underlying exception is a connection error rather than 403/429.
4. Conditionally increase max_retries or use a token with a higher rate-limit tier (GitHub Apps).

Example fix

# before
# 3 parallel scrapers, same GH_TOKEN -> REST failed after 6 retries: 403

# after
# single scraper instance; rerun after rate-limit reset
$ python scrape.py --resume

def rate_budget_ok(remaining: int | None, reset_epoch: int | None, needed: int = 1) -> bool:
    return remaining is None or remaining >= needed or bool(reset_epoch)

try:
    resp = client.rest("GET", "/repos/o/r/issues")
except RuntimeError as e:
    if "REST failed after" in str(e) and "403" in str(e.__cause__ or ""):
        wait_for_rate_limit_reset(client.rest_reset)  # sleep until reset epoch, rerun
    else:
        raise

When it happens

Trigger: Sustained rate limiting that outlasts the retry budget (repeated 403/429), or persistent network failures (connection refused, DNS, proxy) on every REST attempt for a paginated listing call.

Common situations: Scraping very large repos beyond the rate-limit budget (unauthenticated-range limits despite a token; secondary/abuse rate limits); running many parallel scraper instances sharing one token; network egress blocked in CI.

Related errors

• GraphQL failed after {max_retries} retries: {last_err}
• VirusTotal upload failed after {attempts} attempt(s): {last_
• Could not refresh ChatGPT authorization. Please retry.
• Could not reach ChatGPT Codex.
• llama-server embedder POST {path} failed after retry

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/3b9286a8c0519a5f. Report an issue: GitHub.