ScrapeGraphAI/Scrapegraph-ai · error · FreeProxyException

missing proxy servers for criteria

Error message

missing proxy servers for criteria

What it means

FreeProxyException from search_all in the proxy-rotation utility: the proxy search (via proxybroker or the free-proxy fallback) found zero servers matching the configured criteria (protocol/ anonymity/location/etc.).

Source

Thrown at scrapegraphai/utils/proxy_rotation.py:127

                return list(positive)

            except requests.exceptions.RequestException:
                continue

        n = len(positive)

        if n < k and search_outside:
            proxybroker.country_id = None

            try:
                negative = set(search_all(proxybroker, k - n, False))
            except FreeProxyException:
                negative = set()

            positive = positive | negative

        if not positive:
            raise FreeProxyException("missing proxy servers for criteria")

        return list(positive)

    return search_all(proxybroker, max_shape, search_outside_if_empty)


def _parse_proxy(proxy: ProxySettings) -> ProxySettings:
    """parses a proxy configuration with known server

    Args:
        proxy: The proxy configuration to parse.

    Returns:
        A 'playwright' compliant proxy configuration.
    """
    assert "server" in proxy, "missing server in the proxy configuration"

    auhtorization = [x in proxy for x in ("username", "password")]

View on GitHub (pinned to 532dfffbf6)

Solutions

1. Relax criteria (drop country/anonymity filters or allow HTTP)
2. Pass specific proxy servers in config instead of searching free ones for production use
3. Catch FreeProxyException and retry with search_outside_if_empty=True or fall back to direct connection
4. Check network egress / DNS if searches consistently return nothing

Example fix

# before
config = {"proxy": {"search": True, "criteria": {"protocol": "https", "country": "de"}}}
# after
config = {"proxy": {"server": "http://user:pass@gate.nodemaven.com:8080"}}

try:
    proxies = search_proxy_servers(criteria)
except FreeProxyException:
    proxies = []

from scrapegraphai.utils.proxy_rotation import search_proxy_servers
from proxy_rotation_lib import FreeProxyException
try:
    proxies = search_proxy_servers(criteria)
except FreeProxyException as e:
    logger.warning("no free proxies: %s; going direct", e)
    proxies = []  # skip proxy rotation

When it happens

Trigger: Calling search_proxy_servers/search_all with restrictive criteria (e.g. HTTPS + specific country + high anonymity) when the search returns no matches; also raised when search_outside_if_empty finds nothing outside the criteria.

Common situations: Rotating free proxies with very narrow filters; upstream proxy lists temporarily depleted or blocked; network egress blocked so all lookups fail.

Related errors

• Invalid proxy server format: {proxy['server']}
• Search request timed out after {timeout} seconds
• Search request failed: {str(e)}
• DuckDuckGo search failed: {str(e)}
• SearXNG search failed: {str(e)}

AI-assisted analysis of ScrapeGraphAI/Scrapegraph-ai@532dfffbf6 (2026-08-28). Data as JSON: /api/errors/870403a49e5ff483. Report an issue: GitHub.