unclecode/crawl4ai · warning · ValueError

Invalid proxy string format: {proxy_str}

Error message

Invalid proxy string format: {proxy_str}

What it means

EgressBlocked ('URL blocked') from resolve_and_pin's name-based pre-check (the pin path's mirror of assert_host_allowed): the hostname is blocklisted or starts with 'host.docker.internal'. Even though pinning happens in this function, container-internal names are refused by name before DNS to prevent reaching host services.

Source

Thrown at crawl4ai/async_configs.py:613

            if ":" in credentials:
                username, password = credentials.split(":", 1)
                return ProxyConfig(
                    server=f"{protocol}://{server_part}",
                    username=username,
                    password=password,
                )
        # URL without credentials (keep scheme)
        if "://" in s and "@" not in s:
            return ProxyConfig(server=s)
        # Colon separated forms
        parts = s.split(":")
        if len(parts) == 4:
            ip, port, username, password = parts
            return ProxyConfig(server=f"http://{ip}:{port}", username=username, password=password)
        if len(parts) == 2:
            ip, port = parts
            return ProxyConfig(server=f"http://{ip}:{port}")
        raise ValueError(f"Invalid proxy string format: {proxy_str}")
    
    @staticmethod
    def from_dict(proxy_dict: Dict) -> "ProxyConfig":
        """Create a ProxyConfig from a dictionary."""
        return ProxyConfig(
            server=proxy_dict.get("server"),
            username=proxy_dict.get("username"),
            password=proxy_dict.get("password"),
            ip=proxy_dict.get("ip"),
        )
    
    @staticmethod
    def from_env(env_var: str = "PROXIES") -> List["ProxyConfig"]:
        """Load proxies from environment variable.
        
        Args:
            env_var: Name of environment variable containing comma-separated proxy strings
            

View on GitHub (pinned to 7e80152142)

Solutions

1. Serve the target through a public hostname instead of the docker-internal one
2. Fix the origin app so it emits externally routable self-URLs (public APP_URL/base-url setting)
3. Use an explicitly ALLOW_INTERNAL deployment for internal-target crawl jobs

Example fix

# before (app inside compose emits internal self-links)
redirect Location: http://host.docker.internal:3000/login

# after (configure the app's public base URL)
redirect Location: https://app.example.com/login

def redirect_location_safe(location: str) -> bool:
    h = (urlparse(location).hostname or "").lower()
    return bool(h) and not h.startswith("host.docker.internal") and h not in BLOCKED_HOSTNAMES

from egress_broker import EgressBlocked, check_redirect
try:
    target = check_redirect(location)
except EgressBlocked:
    stop_chain(location)  # never follow internal-name redirects

When it happens

Trigger: Crawling (or following a redirect to) a URL whose host is in _BLOCKED_HOSTNAMES or is host.docker.internal*, while ALLOW_INTERNAL is off — check_redirect delegates here, so a redirect Location pointing at the docker host is equally blocked.

Common situations: A crawled page linking to host.docker.internal (common in apps that generate self-referential container URLs); redirects from a public site to an internal name during misconfigured deployments; blocklist additions catching previously allowed names.

Related errors

• Unsupported import format: {format}
• Unsupported export format: {format}
• File not found: {filepath}
• Invalid parameter(s) for {klass.__name__}: {invalid}
• field '{key}' is not permitted on {type_name} from an untrus

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/b171d48578a0a029. Report an issue: GitHub.