nexu-io/open-design · error · RedditRateLimitError

Reddit rate limited (429) fetching {url}

Error message

Reddit rate limited (429) fetching {url}

What it means

RedditRateLimitError raised in fetch_thread_data when http.get_reddit_json raises an HTTPError with status_code 429. Unlike other HTTP errors (which return None and are silently tolerated), 429 is re-thrown so the caller can bail out of Reddit enrichment entirely rather than hammering a rate-limited endpoint.

Source

Thrown at design-templates/last30days/scripts/lib/reddit_enrich.py:67

    Returns:
        Thread data dict or None on failure

    Raises:
        RedditRateLimitError: When Reddit returns 429 (caller should bail)
    """
    if mock_data is not None:
        return mock_data

    path = extract_reddit_path(url)
    if not path:
        return None

    try:
        data = http.get_reddit_json(path, timeout=timeout, retries=retries)
        return data
    except http.HTTPError as e:
        if e.status_code == 429:
            raise RedditRateLimitError(f"Reddit rate limited (429) fetching {url}") from e
        return None


def parse_thread_data(data: Any) -> Dict[str, Any]:
    """Parse Reddit thread JSON into structured data.

    Args:
        data: Raw Reddit JSON response

    Returns:
        Dict with submission and comments data
    """
    result = {
        "submission": None,
        "comments": [],
    }

    if not isinstance(data, list) or len(data) < 1:

View on GitHub (pinned to 5be4028344)

Solutions

1. Set SCRAPECREATORS_API_KEY so the pipeline prefers the rate-limit-free ScrapeCreators Reddit backend.
2. Catch RedditRateLimitError and back off (exponential sleep) or skip Reddit enrichment for this run.
3. Reduce concurrency / thread volume when using the free reddit.com/.json backend.

Example fix

# before
try:
    data = fetch_thread_data(url)
except RedditRateLimitError:
    pass  # swallowed, next call also 429s

# after
from lib.reddit_enrich import RedditRateLimitError
import time
try:
    data = fetch_thread_data(url)
except RedditRateLimitError:
    time.sleep(60)
    raise

null

null

from lib.reddit_enrich import RedditRateLimitError, fetch_thread_data

try:
    data = fetch_thread_data(url)
except RedditRateLimitError:
    # Reddit is throttled; stop enriching Reddit for this run.
    data = None

When it happens

Trigger: Calling fetch_thread_data on a reddit.com/.json URL when Reddit's edge returns 429 (Too Many Requests). This backend is the free fallback; the preferred ScrapeCreators backend does not rate-limit this way.

Common situations: Bulk-enriching many Reddit threads without the ScrapeCreators API key. Running from a shared IP that Reddit has throttled. Bursty concurrent calls exceeding Reddit's anonymous rate budget.

Related errors

• collab cloud error ${status} (${code})
• elevenlabs voices ${resp.status}: ${errText.slice(0, 240)}
• public_github_repository_metric request failed with ${respon
• daemon ${resp.status} on ${url}: ${body || resp.statusText}
• anthropic ${resp.status}: ${await resp.text().catch(() => ''

AI-assisted analysis of nexu-io/open-design@5be4028344 (2026-08-12). Data as JSON: /api/errors/3d94b7af8757c41b. Report an issue: GitHub.