D4Vinci/Scrapling · error · ValueError

Proxy dict must have a 'server' key

Error message

Proxy dict must have a 'server' key

What it means

Raised during ProxyRotator construction when an element of the `proxies` list is a dict that lacks the required 'server' key. Playwright-style proxy dicts are keyed on 'server' (+ optional 'username'/'password'), and the rotator refuses dicts that omit it.

Source

Thrown at scrapling/engines/toolbelt/proxy_rotation.py:79

            - Dict format: {"server": "http://proxy:8080", "username": "user", "password": "pass"}
        :param strategy: Rotation strategy function. Takes (proxies, current_index) and returns (proxy, next_index). Defaults to cyclic_rotation.
        """
        if not proxies:
            raise ValueError("At least one proxy must be provided")

        if not callable(strategy):
            raise TypeError(f"strategy must be callable, got {type(strategy).__name__}")

        self._strategy = strategy
        self._lock = Lock()

        # Validate and store proxies
        self._proxies: List[ProxyType] = []
        self._proxy_to_index: Dict[str, int] = {}  # O(1) lookup by unique key (server + username)
        for i, proxy in enumerate(proxies):
            if isinstance(proxy, (str, dict)):
                if isinstance(proxy, dict) and "server" not in proxy:
                    raise ValueError("Proxy dict must have a 'server' key")

                self._proxy_to_index[_get_proxy_key(proxy)] = i
                self._proxies.append(proxy)
            else:
                raise TypeError(f"Invalid proxy type: {type(proxy)}. Expected str or dict.")

        self._current_index = 0

    def get_proxy(self) -> ProxyType:
        """Get the next proxy according to the rotation strategy."""
        with self._lock:
            proxy, self._current_index = self._strategy(self._proxies, self._current_index)
            return proxy

    @property
    def proxies(self) -> List[ProxyType]:
        """Get a copy of all configured proxies."""
        return list(self._proxies)

View on GitHub (pinned to 5d213a2d47)

Solutions

  1. Rename the key to 'server': {'server': 'http://p:8080', 'username': ..., 'password': ...}.
  2. Normalize each dict before building the rotator: {'server': d.get('server') or d.get('host'), ...}.
  3. Skip or log malformed entries while building the pool instead of passing them through.

Example fix

# before
pool = [{"host": "http://p1:8080"}, "http://p2:8080"]
rotator = ProxyRotator(pool)

# after
pool = [{"server": "http://p1:8080"}, "http://p2:8080"]
rotator = ProxyRotator(pool)
Defensive patterns

Strategy: validation

Validate before calling

def has_server_key(p) -> bool:
    return not isinstance(p, dict) or "server" in p

bad = [p for p in pool if isinstance(p, dict) and "server" not in p]
assert not bad, f"proxy dicts missing 'server': {bad}"

Type guard

def is_playwright_proxy_dict(value: object) -> bool:
    return isinstance(value, dict) and isinstance(value.get("server"), str)

Try / catch

try:
    rotator = ProxyRotator(proxies=pool)
except ValueError as e:
    if "'server' key" in str(e):
        pool = [p if not isinstance(p, dict) else {**{'server': p.get('server') or p.get('host', '')}, **p} for p in pool]
        rotator = ProxyRotator(proxies=pool)
    else:
        raise

Prevention

When it happens

Trigger: ProxyRotator(proxies=[{'host': 'http://p:8080'}]), [{'server': ...}, {'username': 'u', 'password': 'p'}] (second entry missing server), or a dict produced by dropping empty values: {k: v for k, v in d.items() if v} where server was empty string.

Common situations: Mixed-quality proxy pools merged from multiple vendors where one source uses 'host'/'address' instead of 'server'; dict comprehensions that filter out falsy values; hand-written configs with typos in the key.

Related errors

AI-assisted analysis of D4Vinci/Scrapling@5d213a2d47 (2026-08-14). Data as JSON: /api/errors/8dc17f276ed06877. Report an issue: GitHub.