unclecode/crawl4ai · error · RuntimeError

Cannot create new browser contexts when using use_persistent

Error message

Cannot create new browser contexts when using use_persistent_context=True. Persistent context uses a single shared context.

What it means

create_browser_context() refuses to make new contexts when self.browser is None AND the manager was launched with use_persistent_context=True (flag _launched_persistent). A persistent-context Playwright launch returns a single shared context instead of a Browser object, so new-context creation is structurally impossible.

Source

Thrown at crawl4ai/browser_manager.py:1263

            """)
            context._crawl4ai_shadow_dom_injected = True

        # Apply custom init_scripts from BrowserConfig (for stealth evasions, etc.)
        if self.config.init_scripts:
            for script in self.config.init_scripts:
                await context.add_init_script(script)

    async def create_browser_context(self, crawlerRunConfig: CrawlerRunConfig = None):
        """
        Creates and returns a new browser context with configured settings.
        Applies text-only mode settings if text_mode is enabled in config.

        Returns:
            Context: Browser context object with the specified configurations
        """
        if self.browser is None:
            if self._launched_persistent:
                raise RuntimeError(
                    "Cannot create new browser contexts when using "
                    "use_persistent_context=True. Persistent context uses a "
                    "single shared context."
                )
            raise RuntimeError(
                "Browser is not available. It may have been closed, crashed, "
                "or not yet started. Ensure the browser is running before "
                "creating new contexts."
            )
        # Base settings
        user_agent = self.config.headers.get("User-Agent", self.config.user_agent) 
        viewport_settings = {
            "width": self.config.viewport_width,
            "height": self.config.viewport_height,
        }
        proxy_settings = {"server": self.config.proxy} if self.config.proxy else None

        # CSS extensions (blocked separately via avoid_css flag)

View on GitHub (pinned to 7e80152142)

Solutions

1. Reuse the existing persistent context instead of creating new ones: access crawler.crawler_strategy.browser_context or the shared context rather than calling create_browser_context().
2. If you need isolated contexts, disable persistence: BrowserConfig(use_persistent_context=False) and store auth state with storage_state instead.
3. Run one crawler at a time per persistent profile.

Example fix

# before
BrowserConfig(use_persistent_context=True)
# ... later: ctx = await manager.create_browser_context()  # RuntimeError

# after
BrowserConfig(use_persistent_context=False, storage_state='auth.json')
ctx = await manager.create_browser_context()  # works

if browser_config.use_persistent_context and need_isolated_contexts:
    raise ValueError('use_persistent_context cannot be combined with per-session contexts; use storage_state instead')

def can_create_contexts(browser_config) -> bool:
    return not getattr(browser_config, 'use_persistent_context', False)

try:
    ctx = await manager.create_browser_context(cfg)
except RuntimeError as e:
    if 'use_persistent_context=True' in str(e):
        ctx = manager.browser_context  # reuse the shared persistent context
    else:
        raise

When it happens

Trigger: BrowserConfig(use_persistent_context=True) combined with session-per-crawl behavior that calls create_browser_context() for each new session (e.g. multiple concurrent crawlers sharing one manager, or explicitly requesting a new context).

Common situations: Enabling persistent contexts for logged-in sessions then attempting parallel crawls that each want isolated contexts; library code (deep crawling, dispatcher) that assumes context creation is always available.

Related errors

• Timeout after {timeout}ms waiting for selector '{css_selecto
• Invalid CSS selector: '{css_selector}'
• [NSTProxy] Invalid protocol: {protocol}
• concurrency must be positive
• timeout must be positive

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/abbe0828ab6614d1. Report an issue: GitHub.