NanmiCoder/MediaCrawler · error · DataFetchError

get containerid failed

Error message

get containerid failed

What it means

Raised by WeiboClient.get_container_id when the GET /u/{creator_id} profile page response contains no M_WEIBOCN_PARAMS cookie. That cookie is the only source of the fid/lfid container IDs that the detail and list APIs require, so its absence means the profile page was not served normally (redirected to a verification/login page). It is a DataFetchError.

Source

Thrown at media_platform/weibo/client.py:322

            except httpx.HTTPError as exc:  # some wrong when call httpx.request method, such as connection error, client error, server error or response status code is not 2xx
                utils.logger.error(f"[DouYinClient.get_aweme_media] {exc.__class__.__name__} for {exc.request.url} - {exc}")    # Keep original exception type name for developer debugging
                return None

    async def get_creator_container_info(self, creator_id: str) -> Dict:
        """
        Get user's container ID, container information represents the real API request path
            fid_container_id: Container ID for user's Weibo detail API
            lfid_container_id: Container ID for user's Weibo list API
        Args:
            creator_id: User ID

        Returns: Dictionary with container IDs

        """
        response = await self.get(f"/u/{creator_id}", return_response=True)
        m_weibocn_params = response.cookies.get("M_WEIBOCN_PARAMS")
        if not m_weibocn_params:
            raise DataFetchError("get containerid failed")
        m_weibocn_params_dict = parse_qs(unquote(m_weibocn_params))
        return {"fid_container_id": m_weibocn_params_dict.get("fid", [""])[0], "lfid_container_id": m_weibocn_params_dict.get("lfid", [""])[0]}

    async def get_creator_info_by_id(self, creator_id: str) -> Dict:
        """
        Get user details by user ID
        Args:
            creator_id:

        Returns:

        """
        uri = "/api/container/getIndex"
        containerid = f"100505{creator_id}"
        params = {
            "jumpfrom": "weibocom",
            "type": "uid",
            "value": creator_id,

View on GitHub (pinned to d6f7c5bb90)

Solutions

1. Verify the creator_id values in WEIBO_CREATOR_ID_LIST are real, active weibo UIDs (open https://weibo.com/u/{id} in a browser).
2. Re-login to weibo to refresh cookies before crawling creators.
3. Enable the proxy pool or slow down requests so the profile page is served normally.
4. Wrap the call and skip creators that consistently fail rather than aborting the whole creator loop.

Example fix

// before
container = await wb_client.get_container_id(creator_id)
// after
try:
    container = await wb_client.get_container_id(creator_id)
except DataFetchError:
    utils.logger.warning(f"creator {creator_id} unreachable, skipping")
    continue

async def can_get_container(wb_client, creator_id: str) -> bool:
    try:
        await wb_client.get_container_id(creator_id)
        return True
    except DataFetchError:
        return False

for user_id in config.WEIBO_CREATOR_ID_LIST:
    try:
        container = await wb_client.get_container_id(user_id)
    except DataFetchError as e:
        utils.logger.warning(f"creator {user_id}: {e}; skipping")
        continue

When it happens

Trigger: Fetching a weibo user profile while not logged in or with expired cookies, an invalid/deactivated creator_id in WEIBO_CREATOR_ID_LIST, or an anti-bot response that sets no M_WEIBOCN_PARAMS cookie.

Common situations: WEIBO_CREATOR_ID_LIST contains a wrong or banned UID; the shared cookie string in config has gone stale; the crawler IP is flagged and weibo serves a security page instead of the profile.

Related errors

• Get creator info error
• get response code error: {response.status_code}
• get weibo detail err: {response.text}
• Unable to parse creator ID from URL: {url}
• Unable to parse creator ID from URL: {url}

AI-assisted analysis of NanmiCoder/MediaCrawler@d6f7c5bb90 (2026-08-15). Data as JSON: /api/errors/637dbff24562033d. Report an issue: GitHub.