{"record":{"id":"fea5fb8f08290986","repo":"mvanhorn/last30days-skill","slug":"reddit-public-search-failed-exc","errorCode":null,"errorMessage":"Reddit public search failed: {exc}","messagePattern":"Reddit public search failed: (.+?)","errorType":"exception","errorClass":"SourceRunError","httpStatus":null,"severity":"error","filePath":"skills/last30days/scripts/lib/pipeline.py","lineNumber":4155,"sourceCode":"            min_items = int(config.get(env.REDDIT_SC_MIN_ITEMS_VAR) or 0)\n        except (TypeError, ValueError):\n            min_items = 0\n        public_results: list[dict] = []\n        public_failure: Exception | None = None\n        try:\n            public_results = reddit_public.search_reddit_public(\n                reddit_query, from_date, to_date, depth=depth,\n                subreddits=subreddits, dedicated_subreddits=dedicated_subreddits,\n            ) or []\n        except Exception as exc:\n            public_failure = exc\n            sys.stderr.write(\n                f\"[Reddit] Public search failed ({type(exc).__name__}: {exc})\"\n            )\n            if not has_sc_key:\n                sys.stderr.write(\"\\n\")\n                state = reddit.classify_run_failure(str(exc))\n                raise SourceRunError(f\"Reddit public search failed: {exc}\", state) from exc\n            sys.stderr.write(\", using ScrapeCreators backup\\n\")\n        # Enough free results, or no key to backfill with -> done. max(min_items,\n        # 1) keeps the default (min_items=0) as empty-only AND treats exactly\n        # `min_items` results as acceptable (no backfill) for min_items > 0.\n        if len(public_results) >= max(min_items, 1) or not has_sc_key:\n            return public_results, {}\n        if public_results:\n            sys.stderr.write(\n                f\"[Reddit] Free path returned {len(public_results)} \"\n                f\"(below the {min_items}-item floor); backfilling with ScrapeCreators\\n\"\n            )\n        try:\n            result = reddit.search_and_enrich(\n                reddit_query, from_date, to_date, depth=depth,\n                token=config.get(\"SCRAPECREATORS_API_KEY\"),\n                subreddits=subreddits,\n            )\n            sc_items = reddit.parse_reddit_response(result)","sourceCodeStart":4137,"sourceCodeEnd":4173,"githubUrl":"https://github.com/mvanhorn/last30days-skill/blob/c7460f6114449ddfe6ea3fc2f23c3d910c0e740c/skills/last30days/scripts/lib/pipeline.py#L4137-L4173","documentation":"SourceRunError raised when the free/public Reddit search path (reddit_public.search_reddit_public) raises AND the caller has no ScrapeCreators key to backfill with (has_sc_key is false). Since there is no fallback, the original exception is classified via reddit.classify_run_failure and re-raised as a SourceRunError with the underlying message chained (`from exc`).","triggerScenarios":"No SCRAPECREATORS_API_KEY in config, so the run uses public Reddit first; search_reddit_public raises (HTTP error, parse failure, network timeout); has_sc_key is False so no backfill is attempted and the error propagates immediately.","commonSituations":"Keyless installs relying on the free Reddit path; Reddit endpoint changes or HTML restructuring breaking the public client; host IP rate-limited/blocked by Reddit; transient network failures behind corporate proxies.","solutions":["Retry after a short backoff — public Reddit failures are often transient rate limits.","Set SCRAPECREATORS_API_KEY so the pipeline can fall back to the keyed path when public search fails.","Check whether Reddit is reachable from the host (curl a reddit.com search URL) and fix proxy/firewall issues.","Classify via exc.state: AUTH/parse states indicate client bugs or blocks, RATE_LIMITED indicates backoff needed."],"exampleFix":"# before: keyless run, public Reddit breaks, whole source fails\nresults = run_pipeline(topic, requested_sources=[\"reddit\"], config={})\n\n# after: provide a key so the pipeline can backfill past public failures\nresults = run_pipeline(topic, requested_sources=[\"reddit\"], config={\"SCRAPECREATORS_API_KEY\": os.environ[\"SCRAPECREATORS_API_KEY\"]})","handlingStrategy":"fallback","validationCode":"has_key = bool(config.get(\"SCRAPECREATORS_API_KEY\"))\nif not has_key and \"reddit\" in requested_sources:\n    # keyless run: public path is the ONLY path; expect hard failures on blocks\n    log(\"reddit running without ScrapeCreators backfill\")","typeGuard":null,"tryCatchPattern":"try:\n    items, meta = fetch_reddit(...)\nexcept SourceRunError as exc:\n    if \"public search failed\" in str(exc) and not has_key:\n        items, meta = [], {}  # degrade gracefully, other sources carry the run\n    else:\n        raise","preventionTips":["Set SCRAPECREATORS_API_KEY even if you usually rely on the free path — it converts hard failures into backfills.","Space out keyless runs to avoid Reddit IP rate limits."],"tags":["reddit","network","api-key","fallback","source-failure"],"backgroundTag":null,"analyzedSha":"c7460f6114449ddfe6ea3fc2f23c3d910c0e740c","analyzedAt":"2026-08-15T03:34:49.540Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}