ScrapeGraphAI/Scrapegraph-ai · error · SearchRequestError

SearXNG search failed: {str(e)}

Error message

SearXNG search failed: {str(e)}

What it means

Raised by the SearXNG backend when querying the SearXNG instance fails: connection refused, non-2xx status (raise_for_status), or invalid JSON. Wrapped into SearchRequestError with the original message.

Source

Thrown at scrapegraphai/utils/research_web.py:370

        "time_range": "",
        "engines": "duckduckgo,bing,brave",
        "results": max_results,
    }

    try:
        response = requests.get(
            f"http://localhost:{port}/search",
            params=params,
            headers=headers,
            timeout=timeout,
        )
        response.raise_for_status()

        json_data = response.json()
        results = [result["url"] for result in json_data.get("results", [])]
        return results[:max_results]
    except Exception as e:
        raise SearchRequestError(f"SearXNG search failed: {str(e)}")


def _search_serper(
    query: str, max_results: int, api_key: str, timeout: int
) -> List[str]:
    """
    Helper function for Serper search.

    Args:
        query (str): Search query
        max_results (int): Maximum number of results to return
        api_key (str): API key for Serper
        timeout (int): Request timeout in seconds

    Returns:
        List[str]: List of URLs from search results
    """
    if not api_key:

View on GitHub (pinned to 532dfffbf6)

Solutions

  1. Verify the SearXNG instance is up (curl its /search endpoint with format=json)
  2. Fix the port/host configuration
  3. Enable JSON output on the SearXNG instance (json format must be allowed)
  4. Retry with backoff for transient instance overload

Example fix

// before
config.search_engine = "searxng"  # instance down -> SearchRequestError
// after
# start instance first: docker run -p 8888:8080 searxng/searxng
config.search_engine = "searxng"; config.port = 8888
Defensive patterns

Strategy: validation

Validate before calling

import requests
def searxng_up(port):
    try:
        r = requests.get(f"http://localhost:{port}/search", params={"q": "test", "format": "json"}, timeout=5)
        return r.ok
    except requests.RequestException:
        return False
if config.search_engine == "searxng" and not searxng_up(config.port):
    raise RuntimeError("SearXNG instance not reachable")

Try / catch

try:
    results = search_on_web(config)
except SearchRequestError as e:
    logger.error("searxng failed: %s", e); results = []

Prevention

When it happens

Trigger: Calling search_on_web with engine='searxng' when the SearXNG instance at the configured port is not running, returns an error status, or responds with non-JSON content.

Common situations: SearXNG Docker container not started or crashed; wrong port in config; instance blocking external clients; JSON format disabled on the instance.

Related errors

AI-assisted analysis of ScrapeGraphAI/Scrapegraph-ai@532dfffbf6 (2026-08-28). Data as JSON: /api/errors/ecacefe4cc80ae5a. Report an issue: GitHub.