playwright_page is required for browser-based comment fetchi

Error message

playwright_page is required for browser-based comment fetching

What it means

Raised by BaiduTieBaClient.get_note_all_comments (media_platform/tieba/client.py:471) when self.playwright_page is None. Comment pagination is fetched by loading the post page in a browser for each comment page, so the method hard-requires a live page. The guard triggers before the pagination while-loop starts.

Source

Thrown at media_platform/tieba/client.py:471

        self,
        note_detail: TiebaNote,
        crawl_interval: float = 1.0,
        callback: Optional[Callable] = None,
        max_count: int = 10,
    ) -> List[TiebaComment]:
        """
        Get all first-level comments for specified post (uses Playwright to access page, avoiding API detection)
        Args:
            note_detail: Post detail object
            crawl_interval: Crawl delay interval in seconds
            callback: Callback function after one post crawl completes
            max_count: Maximum number of comments to crawl per post
        Returns:
            List[TiebaComment]: Comment list
        """
        if not self.playwright_page:
            utils.logger.error("[BaiduTieBaClient.get_note_all_comments] playwright_page is None, cannot use browser mode")
            raise Exception("playwright_page is required for browser-based comment fetching")

        result: List[TiebaComment] = []
        current_page = 1

        while note_detail.total_replay_page >= current_page and len(result) < max_count:
            utils.logger.info(
                f"[BaiduTieBaClient.get_note_all_comments] Accessing comment API, "
                f"note_id: {note_detail.note_id}, page: {current_page}"
            )

            try:
                api_data = await self._get_pc_page_data(note_id=note_detail.note_id, page=current_page)
                comments = self._page_extractor.extract_tieba_note_parent_comments_from_api(
                    api_data, note_detail=note_detail
                )

                if not comments:
                    utils.logger.info(f"[BaiduTieBaClient.get_note_all_comments] Page {current_page} has no comments, stopping crawl")

View on GitHub (pinned to d6f7c5bb90)

Solutions

Attach a Playwright page to the client before starting comment crawls
Run the platform's login/browser bootstrap first so playwright_page is populated
Skip comment crawling (or gate it on bool(client.playwright_page)) in pipelines that legitimately run browserless

Example fix

# before
comments = await client.get_note_all_comments(note)

# after
if not client.playwright_page:
    client.playwright_page = await context.new_page()
comments = await client.get_note_all_comments(note)

Defensive patterns

Strategy: type-guard

Validate before calling

if not client.playwright_page:
    client.playwright_page = await browser_context.new_page()

Type guard

def can_fetch_comments(client) -> bool:
    return client.playwright_page is not None and not client.playwright_page.is_closed()

Prevention

Gate comment crawling on bool(client.playwright_page) in browserless pipelines
Complete login (which creates the page) before the comment stage
Handle browser crashes by re-initializing context and page together

When it happens

Trigger: Calling get_note_all_comments(note_detail, ...) on a client with no playwright_page — e.g. a comment crawl scheduled before browser setup, or a client built only for cookie/HTTP usage.

Common situations: Reusing a client instance after the browser closed; enabling comment crawling (ENABLE_GET_COMMENTS) in a pipeline that never opens a browser; unit tests calling the method directly.

Related errors

AI-assisted analysis of NanmiCoder/MediaCrawler@d6f7c5bb90 (2026-08-15). Data as JSON: /api/errors/f5d815f74f7c50cb. Report an issue: GitHub.