NanmiCoder/MediaCrawler · error

playwright_page is required for browser-based creator notes

Error message

playwright_page is required for browser-based creator notes fetching

What it means

Raised by BaiduTieBaClient.get_notes_by_creator_portrait (media_platform/tieba/client.py:692) when self.playwright_page is None. The creator thread feed (/c/u/feed/myThread) is fetched through the browser session to inherit cookies and defeat API detection, so a live Playwright page is required. It is the feed-level guard; the caller get_all_notes_by_creator_url passes the portrait in after its own extraction.

Source

Thrown at media_platform/tieba/client.py:692

                    "_client_type": "20",
                },
                use_sign=True,
            )
            return self._page_extractor.extract_creator_info_from_api(api_data)

        except Exception as e:
            utils.logger.error(f"[BaiduTieBaClient.get_creator_info_by_url] Failed to get creator info: {e}")
            raise

    async def get_notes_by_creator_portrait(
        self, portrait: str, page_number: int, page_size: int = 20
    ) -> Dict:
        """
        Get creator's thread feed by creator portrait from current PC JSON API.
        """
        if not self.playwright_page:
            utils.logger.error("[BaiduTieBaClient.get_notes_by_creator_portrait] playwright_page is None, cannot use browser mode")
            raise Exception("playwright_page is required for browser-based creator notes fetching")

        utils.logger.info(
            f"[BaiduTieBaClient.get_notes_by_creator_portrait] Accessing creator feed API, "
            f"portrait: {portrait}, page: {page_number}"
        )
        return await self._fetch_json_by_browser(
            "/c/u/feed/myThread",
            params={
                "pn": page_number,
                "rn": page_size,
                "portrait": portrait,
                "type": 1,
                "un": "",
                "subapp_type": "pc",
                "_client_type": "20",
            },
            use_sign=True,
        )

View on GitHub (pinned to d6f7c5bb90)

Solutions

  1. Run the standard browser+login bootstrap so the client holds a live page before creator feed calls
  2. Guard caller code: skip or re-initialize the browser when client.playwright_page is missing
  3. Ensure Playwright and its Chromium build are installed in the deployment environment
Defensive patterns

Strategy: type-guard

Validate before calling

if not client.playwright_page or client.playwright_page.is_closed():
    client.playwright_page = await browser_context.new_page()

Type guard

def feed_fetch_ready(client) -> bool:
    p = getattr(client, "playwright_page", None)
    return p is not None and not p.is_closed()

Prevention

When it happens

Trigger: Calling get_notes_by_creator_portrait(portrait, page_number) directly on a client without a page, or reaching it via get_all_notes_by_creator_url when the browser was never started/closed.

Common situations: Creator crawls launched before login completes; browser crash or manual close mid-crawl leaving playwright_page unset/stale; scripts calling the low-level API method without the standard crawler bootstrap.

Related errors


AI-assisted analysis of NanmiCoder/MediaCrawler@d6f7c5bb90 (2026-08-15). Data as JSON: /api/errors/005783263e9b1704. Report an issue: GitHub.