{"record":{"id":"579ff6fd692730b5","repo":"ScrapeGraphAI/Scrapegraph-ai","slug":"serper-search-failed-str-e","errorCode":null,"errorMessage":"Serper search failed: {str(e)}","messagePattern":"Serper search failed: (.+?)","errorType":"exception","errorClass":"SearchRequestError","httpStatus":null,"severity":"error","filePath":"scrapegraphai/utils/research_web.py","lineNumber":416,"sourceCode":"            json=data,\n            headers=headers,\n            timeout=timeout,\n        )\n        response.raise_for_status()\n\n        json_data = response.json()\n        results = []\n\n        # Extract organic search results\n        for item in json_data.get(\"organic\", []):\n            if \"link\" in item:\n                results.append(item[\"link\"])\n                if len(results) >= max_results:\n                    break\n\n        return results\n    except Exception as e:\n        raise SearchRequestError(f\"Serper search failed: {str(e)}\")\n\n\ndef format_proxy(proxy_config: Union[str, Dict, ProxyConfig]) -> str:\n    \"\"\"\n    Format proxy configuration into a string.\n\n    Args:\n        proxy_config: Proxy configuration as string, dict, or ProxyConfig\n\n    Returns:\n        str: Formatted proxy string\n    \"\"\"\n    if isinstance(proxy_config, str):\n        return proxy_config\n\n    if isinstance(proxy_config, dict):\n        proxy_config = ProxyConfig(**proxy_config)\n","sourceCodeStart":398,"sourceCodeEnd":434,"githubUrl":"https://github.com/ScrapeGraphAI/Scrapegraph-ai/blob/532dfffbf6ee823a6c9cf8cfedc24a93bf026780/scrapegraphai/utils/research_web.py#L398-L434","documentation":"Raised by the Serper backend when the HTTP call or response parsing fails: network error, non-200 response, invalid JSON, or missing 'link' keys in results. Wrapped into SearchRequestError.","triggerScenarios":"Calling search_on_web with engine='serper' when serper.dev returns an error (invalid key, quota exceeded), the request times out at HTTP level, or the JSON payload lacks expected fields.","commonSituations":"Expired or invalid Serper API key (401/403); exhausted free quota; num exceeding API limits; transient serper.dev outages.","solutions":["Check str(e) — 401/403 means bad key, 429 means quota","Validate the API key with a manual curl to google.serper.dev","Add retry with backoff for 5xx/transient failures","Reduce max_results if num is rejected"],"exampleFix":"// before\nresults = search_on_web(config)  # serper 401\n// after\n# verify key: curl -X POST google.serper.dev/search -H 'X-API-KEY: $KEY' ...\nconfig.serper_api_key = os.getenv(\"SERPER_APIKEY\")","handlingStrategy":"retry","validationCode":"import requests\ndef serper_key_ok(key):\n    r = requests.post(\"https://google.serper.dev/search\", headers={\"X-API-KEY\": key}, json={\"q\": \"ping\"}, timeout=10)\n    return r.status_code == 200","typeGuard":null,"tryCatchPattern":"for attempt in range(3):\n    try:\n        results = search_on_web(config); break\n    except SearchRequestError as e:\n        if attempt == 2: raise\n        time.sleep(2 ** attempt)","preventionTips":["Validate the Serper key once at startup","Retry only transient (5xx/network) failures, not 401/403","Monitor quota usage"],"tags":["api","web-search","serper"],"backgroundTag":"api-request-failed","analyzedSha":"532dfffbf6ee823a6c9cf8cfedc24a93bf026780","analyzedAt":"2026-08-28T15:19:38.821Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}