unclecode/crawl4ai · warning · ValueError

Invalid parameter(s) for {klass.__name__}: {invalid}

Error message

Invalid parameter(s) for {klass.__name__}: {invalid}

What it means

EgressBlocked ('URL blocked') from assert_host_allowed when DNS returns an answer whose IP is non-globally-routable — loopback, private (RFC1918), link-local, and embedded IPv4 transition forms all count. The rule is strict: if ANY resolved address is forbidden, the whole host is rejected, closing the DNS-rebinding window where a name resolves to both a public and a private IP.

Source

Thrown at crawl4ai/async_configs.py:87

                    kwargs[key] = copy.deepcopy(value)
        original_init(self, *args, **kwargs)

    cls.__init__ = wrapped_init
    cls._user_defaults = {}

    @classmethod
    def set_defaults(klass, **kwargs):
        """Set class-level default overrides for new instances.

        Args:
            **kwargs: Parameter names and their default values.

        Raises:
            ValueError: If any key is not a valid ``__init__`` parameter.
        """
        invalid = set(kwargs) - valid_params
        if invalid:
            raise ValueError(
                f"Invalid parameter(s) for {klass.__name__}: {invalid}"
            )
        for k, v in kwargs.items():
            klass._user_defaults[k] = copy.deepcopy(v)

    @classmethod
    def get_defaults(klass):
        """Return a deep copy of the current class-level defaults."""
        return copy.deepcopy(klass._user_defaults)

    @classmethod
    def reset_defaults(klass, *names):
        """Clear class-level defaults.

        With no arguments, removes all overrides.
        With arguments, removes only the named overrides.
        """
        if names:

View on GitHub (pinned to 7e80152142)

Solutions

1. Use a hostname whose DNS answers are all globally routable (check: dig +short <host> from the container)
2. If you own the DNS zone, remove the private/loopback records from the public-facing name
3. For intentionally internal targets, use an ALLOW_INTERNAL deployment explicitly scoped for that job

Example fix

# diagnose from inside the container
# dig +short risky.example.com
# 203.0.113.10
# 10.0.0.5      <- this answer triggers EgressBlocked
# fix: publish only public records for that name, then retry the crawl

import socket, ipaddress
def all_answers_global(host: str) -> bool:
    try:
        answers = socket.getaddrinfo(host, 443, proto=socket.IPPROTO_TCP)
    except socket.gaierror:
        return False
    return all(ipaddress.ip_address(a[4][0]).is_global for a in answers)

from egress_broker import EgressBlocked
try:
    assert_host_allowed(host, port)
except EgressBlocked as e:
    if not all_answers_global(host):
        flag_domain_for_dns_review(host)  # split-horizon / rebinding suspect
    raise

When it happens

Trigger: Crawling a public-looking domain whose DNS includes an A/AAAA record for 10.x/172.16-31.x/192.168.x/127.x/169.254.x or an IPv6 with an embedded private v4 form. Multi-answer DNS (round-robin with one internal record) also triggers it.

Common situations: Split-horizon DNS where the same name resolves internally and externally; a domain temporarily publishing an internal IP; trying to reach cloud-metadata-adjacent addresses via a name; test domains pointing at 127.0.0.1.

Related errors

• enable_stealth cannot be used with browser_mode='builtin'. S
• Unsupported export format: {format}
• File not found: {filepath}
• Unsupported import format: {format}
• field '{key}' is not permitted on {type_name} from an untrus

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/5c87c92cb2a1e2b1. Report an issue: GitHub.