mvanhorn/last30days-skill · error · SourceRunError

Reddit public search failed: {exc}

Error message

Reddit public search failed: {exc}

What it means

SourceRunError raised when the free/public Reddit search path (reddit_public.search_reddit_public) raises AND the caller has no ScrapeCreators key to backfill with (has_sc_key is false). Since there is no fallback, the original exception is classified via reddit.classify_run_failure and re-raised as a SourceRunError with the underlying message chained (`from exc`).

Source

Thrown at skills/last30days/scripts/lib/pipeline.py:4155

            min_items = int(config.get(env.REDDIT_SC_MIN_ITEMS_VAR) or 0)
        except (TypeError, ValueError):
            min_items = 0
        public_results: list[dict] = []
        public_failure: Exception | None = None
        try:
            public_results = reddit_public.search_reddit_public(
                reddit_query, from_date, to_date, depth=depth,
                subreddits=subreddits, dedicated_subreddits=dedicated_subreddits,
            ) or []
        except Exception as exc:
            public_failure = exc
            sys.stderr.write(
                f"[Reddit] Public search failed ({type(exc).__name__}: {exc})"
            )
            if not has_sc_key:
                sys.stderr.write("\n")
                state = reddit.classify_run_failure(str(exc))
                raise SourceRunError(f"Reddit public search failed: {exc}", state) from exc
            sys.stderr.write(", using ScrapeCreators backup\n")
        # Enough free results, or no key to backfill with -> done. max(min_items,
        # 1) keeps the default (min_items=0) as empty-only AND treats exactly
        # `min_items` results as acceptable (no backfill) for min_items > 0.
        if len(public_results) >= max(min_items, 1) or not has_sc_key:
            return public_results, {}
        if public_results:
            sys.stderr.write(
                f"[Reddit] Free path returned {len(public_results)} "
                f"(below the {min_items}-item floor); backfilling with ScrapeCreators\n"
            )
        try:
            result = reddit.search_and_enrich(
                reddit_query, from_date, to_date, depth=depth,
                token=config.get("SCRAPECREATORS_API_KEY"),
                subreddits=subreddits,
            )
            sc_items = reddit.parse_reddit_response(result)

View on GitHub (pinned to c7460f6114)

Solutions

1. Retry after a short backoff — public Reddit failures are often transient rate limits.
2. Set SCRAPECREATORS_API_KEY so the pipeline can fall back to the keyed path when public search fails.
3. Check whether Reddit is reachable from the host (curl a reddit.com search URL) and fix proxy/firewall issues.
4. Classify via exc.state: AUTH/parse states indicate client bugs or blocks, RATE_LIMITED indicates backoff needed.

Example fix

# before: keyless run, public Reddit breaks, whole source fails
results = run_pipeline(topic, requested_sources=["reddit"], config={})

# after: provide a key so the pipeline can backfill past public failures
results = run_pipeline(topic, requested_sources=["reddit"], config={"SCRAPECREATORS_API_KEY": os.environ["SCRAPECREATORS_API_KEY"]})

has_key = bool(config.get("SCRAPECREATORS_API_KEY"))
if not has_key and "reddit" in requested_sources:
    # keyless run: public path is the ONLY path; expect hard failures on blocks
    log("reddit running without ScrapeCreators backfill")

try:
    items, meta = fetch_reddit(...)
except SourceRunError as exc:
    if "public search failed" in str(exc) and not has_key:
        items, meta = [], {}  # degrade gracefully, other sources carry the run
    else:
        raise

When it happens

Trigger: No SCRAPECREATORS_API_KEY in config, so the run uses public Reddit first; search_reddit_public raises (HTTP error, parse failure, network timeout); has_sc_key is False so no backfill is attempted and the error propagates immediately.

Common situations: Keyless installs relying on the free Reddit path; Reddit endpoint changes or HTML restructuring breaking the public client; host IP rate-limited/blocked by Reddit; transient network failures behind corporate proxies.

Related errors

• Reddit primary and fallback produced no results after failur
• All X backends failed — {last_error}
• Gemini HTTP {exc.code}: {detail}
• Gemini request failed: {exc}
• BRAVE_API_KEY is required when web_backend='brave'

AI-assisted analysis of mvanhorn/last30days-skill@c7460f6114 (2026-08-15). Data as JSON: /api/errors/fea5fb8f08290986. Report an issue: GitHub.