What causes "Browser not initialized for proxy rotation mode" in D4Vinci/Scrapling?

Always use the session as a context manager and fetch inside the `with` block: `with DynamicSession(...) as s: s.fetch(url, proxy=proxy)`.

D4Vinci/Scrapling · error · RuntimeError

Browser not initialized for proxy rotation mode

Error message

Browser not initialized for proxy rotation mode

What it means

Raised in the sync `_page_generator` of the browser engine base (scrapling/engines/_browsers/_base.py:195). In proxy-rotation mode the engine must create a fresh browser context per proxy via `self.browser.new_context(...)`; if `self.browser` is falsy at that point the generator raises RuntimeError. This is marked `no cover` because it indicates the session's internal startup invariant was broken — a live session always has a browser when a proxy is supplied.

Source

Thrown at scrapling/engines/_browsers/_base.py:195

            ):
                xhr_container.append(finished_response)

        return handle_response

    @contextmanager
    def _page_generator(
        self,
        timeout: int | float,
        extra_headers: Optional[Dict[str, str]],
        disable_resources: bool,
        proxy: Optional[ProxyType] = None,
        blocked_domains: Optional[Set[str]] = None,
    ) -> Generator["PageInfo[Page]", None, None]:
        """Acquire a page - either from persistent context or fresh context with proxy."""
        if proxy:
            # Rotation mode: create fresh context with the provided proxy
            if not self.browser:  # pragma: no cover
                raise RuntimeError("Browser not initialized for proxy rotation mode")
            context_options = self._build_context_with_proxy(proxy)
            context: BrowserContext = self.browser.new_context(**context_options)

            page_info = None
            try:
                context = self._initialize_context(self._config, context)
                page_info = self._get_page(timeout, extra_headers, disable_resources, blocked_domains, context=context)
                yield page_info
            finally:
                if page_info is not None and page_info in self.page_pool.pages:
                    self.page_pool.pages.remove(page_info)
                context.close()
        else:
            # Standard mode: use PagePool with persistent context
            page_info = self._get_page(timeout, extra_headers, disable_resources, blocked_domains)
            try:
                yield page_info
            finally:

View on GitHub (pinned to 5d213a2d47)

Solutions

Always use the session as a context manager and fetch inside the `with` block: `with DynamicSession(...) as s: s.fetch(url, proxy=proxy)`.
Check `session._is_alive` before fetching if you keep long-lived session references.
Verify startup succeeded — wrap `__enter__` in try/except and don't reuse the session after failure.
If you must manage lifecycle manually, call the session's start method before any proxied fetch.

Example fix

# before
session = DynamicSession(proxy_rotator=rotator)
resp = session.fetch(url)  # browser never started

# after
with DynamicSession(proxy_rotator=rotator) as session:
    resp = session.fetch(url)

Defensive patterns

Strategy: type-guard

Validate before calling

from scrapling.engines._browsers._controllers import DynamicSession

def session_ready(session: DynamicSession) -> bool:
    return bool(getattr(session, "_is_alive", False)) and bool(getattr(session, "browser", None))

Type guard

def can_fetch_with_proxy(session) -> bool:
    """True only inside the context manager with a live browser."""
    return bool(getattr(session, 'browser', None)) and bool(getattr(session, '_is_alive', False))

Try / catch

try:
    resp = session.fetch(url, proxy=p)
except RuntimeError as e:
    if 'Browser not initialized' in str(e):
        raise RuntimeError('fetch called outside session context') from e
    raise

Prevention

Always fetch inside `with DynamicSession(...) as session:`.
Never reuse a session after its context manager exits.
Treat startup failures as fatal for that session object; create a new one.

When it happens

Trigger: Calling `fetch(url, proxy=...)` or configuring `proxy_rotator` on a `DynamicSession`/`StealthySession` whose `__enter__`/startup did not run or whose browser was already torn down (used after `__exit__`, or startup failed silently); also reachable by poking internals (`session.browser = None`) before a proxied fetch.

Common situations: Using the session outside its context manager (`session = DynamicSession(...); session.fetch(url, proxy=p)` without `with`); storing the session and fetching after `__exit__` closed the browser; a previous startup exception that left `_is_alive` inconsistent.

Related errors

AI-assisted analysis of D4Vinci/Scrapling@5d213a2d47 (2026-08-14). Data as JSON: /api/errors/05008e0925386581. Report an issue: GitHub.