Z4nzu/hackingtool · warning · RateLimited

GitHub search rate limit reached{wait}

Error message

GitHub search rate limit reached{wait}

What it means

RateLimited is raised by hackingtool's GitHub repository search (_fetch in discover.py:502) when the GitHub Search API responds with HTTP 403 or 429 and the response header x-ratelimit-remaining is exactly "0" — i.e. the caller has exhausted GitHub's primary rate limit for the search endpoint. The library deliberately rethrows this as a distinct exception (instead of returning [] like other failures) so callers can tell 'no results / transient error' apart from 'you must wait'. The message embeds a computed wait hint derived from the x-ratelimit-reset header, e.g. "GitHub search rate limit reached (~58s)".

Source

Thrown at src/hackingtool/discover.py:523

        "Accept": "application/vnd.github+json",
        "X-GitHub-Api-Version": "2022-11-28",
        "User-Agent": _USER_AGENT,
    })
    tok = _token()
    if tok:
        req.add_header("Authorization", f"Bearer {tok}")
    try:
        with urllib.request.urlopen(req, timeout=_TIMEOUT) as resp:
            return json.loads(resp.read().decode("utf-8"))
    except urllib.error.HTTPError as exc:
        if exc.code in (403, 429) and exc.headers.get("x-ratelimit-remaining") == "0":
            reset = exc.headers.get("x-ratelimit-reset", "")
            wait = ""
            try:
                wait = f" (~{max(0, int(reset) - int(time.time()))}s)"
            except ValueError:
                pass
            raise RateLimited(f"GitHub search rate limit reached{wait}") from exc
        raise


def _to_repo(item: dict) -> Repo:
    """Read ONLY the allowlisted fields. Nothing else is touched."""
    lic = item.get("license") or {}
    owner = item.get("owner") or {}
    return Repo(
        full_name=item.get("full_name", ""),
        description=(item.get("description") or "").strip(),
        url=item.get("html_url", ""),
        stars=int(item.get("stargazers_count") or 0),
        forks=int(item.get("forks_count") or 0),
        pushed_at=item.get("pushed_at") or "",
        created_at=item.get("created_at") or "",
        archived=bool(item.get("archived")) or bool(item.get("disabled")),
        fork=bool(item.get("fork")),
        license=(lic.get("spdx_id") or "") if lic.get("spdx_id") != "NOASSERTION" else "",

View on GitHub (pinned to 9b92b6156d)

Solutions

1. Set a GitHub token: export HACKINGTOOL_GITHUB_TOKEN (or GITHUB_TOKEN / GH_TOKEN) — this raises the search limit from 10 to 30 requests/minute; create one at https://github.com/settings/tokens (no scopes needed for public search).
2. Wait out the window shown in the message (the "(~Ns)" suffix is seconds until x-ratelimit-reset) and retry the same query — the 24h on-disk cache means a successful retry costs nothing on repeat.
3. Reduce distinct queries: re-run the same wording instead of new phrasings, since cache hits never touch the network.
4. If the token is set but the error persists, verify it is valid (gh auth status or curl -H "Authorization: Bearer $TOKEN" https://api.github.com/rate_limit) — an invalid token silently downgrades you to the anonymous quota.
5. In scripts/loops, wrap calls in a RateLimited handler that sleeps for the reported seconds and retries once (see tryCatchPattern).

Example fix

# before: unauthenticated, 10 search req/min
$ hackingtool find "xss scanner"
$ hackingtool find "sql injection scanner"
$ hackingtool find "subdomain scanner"   # -> RateLimited: GitHub search rate limit reached (~52s)

# after: authenticated, 30 req/min
$ export HACKINGTOOL_GITHUB_TOKEN=ghp_your_personal_access_token
$ hackingtool find "xss scanner"

import json, os, time, urllib.request

def github_search_quota_left() -> int | None:
    """Check https://api.github.com/rate_limit before searching. None = unknown."""
    tok = (os.environ.get("HACKINGTOOL_GITHUB_TOKEN")
           or os.environ.get("GITHUB_TOKEN")
           or os.environ.get("GH_TOKEN") or "").strip()
    req = urllib.request.Request("https://api.github.com/rate_limit")
    if tok:
        req.add_header("Authorization", f"Bearer {tok}")
    try:
        with urllib.request.urlopen(req, timeout=10) as r:
            core = json.load(r)["resources"]["search"]
            return core["remaining"]
    except Exception:
        return None

if github_search_quota_left() == 0:
    raise SystemExit("search quota exhausted; wait or set HACKINGTOOL_GITHUB_TOKEN")
from hackingtool import discover
discover._search("topic:xss")

from hackingtool import discover

def is_rate_limited(exc: BaseException) -> bool:
    """True if exc is hackingtool's GitHub search rate-limit error."""
    return isinstance(exc, discover.RateLimited)

import re, time
from hackingtool import discover

_WAIT_RX = re.compile(r"~(\d+)s")

try:
    repos = discover._search("topic:security language:python")
except discover.RateLimited as exc:
    m = _WAIT_RX.search(str(exc))
    delay = int(m.group(1)) + 2 if m else 60          # header-derived wait, safe floor
    time.sleep(delay)
    repos = discover._search("topic:security language:python")  # cache makes retry cheap
except Exception:
    repos = []                                         # other failures degrade gracefully
# proceed with repos

When it happens

Trigger: Any call chain that reaches discover._search(query) with a cache miss (the 24h on-disk cache in USER_CONFIG_FILE.parent/cache/find/ is empty or expired for that query): it builds a request to the GitHub search/repositories endpoint with per_page=10, and GitHub replies 403 (unauthenticated) or 429 with x-ratelimit-remaining: 0. Without a token the search limit is only 10 requests/minute, so ~10 distinct queries in a minute trigger it; with a token it is 30/minute. The 403+remaining=0 combination is how GitHub reports exhausted unauthenticated quotas (429 is the authenticated form), which is why both codes are checked in discover.py:516.

Common situations: Running hackingtool find repeatedly with different phrasings in a short session while unauthenticated (most common). CI or shared NAT/proxy egress where the 60-requests/hour anonymous pool is consumed by others on the same IP. Providing a GitHub token that is expired, revoked, or lacks access, causing GitHub to treat requests as anonymous. Clock skew or an old cached process holding a stale token. Multi-user scripts or loops that call the search CLI per-item without backoff.

AI-assisted analysis of Z4nzu/hackingtool@9b92b6156d (2026-08-14). Data as JSON: /api/errors/b13a4a6398c787e9. Report an issue: GitHub.