NanmiCoder/MediaCrawler · critical · PlatformAccessError

XHS request blocked with HTTP {response.status_code}

Error message

XHS request blocked with HTTP {response.status_code}

What it means

Raised by XiaoHongShuClient.request when xiaohongshu answers with HTTP 401, 403, or 429. These statuses mean the platform actively blocked the request: 401 invalid/expired session cookie, 403 forbidden (signature or cookie rejected), 429 rate limited. It is a PlatformAccessError, distinct from data errors, so callers can branch on access-level failures.

Source

Thrown at media_platform/xhs/client.py:154

        Wrapper for httpx common request method, processes request response
        Args:
            method: Request method
            url: Request URL
            **kwargs: Other request parameters, such as headers, body, etc.

        Returns:

        """
        # Check if proxy is expired before each request
        await self._refresh_proxy_if_expired()

        # return response.text
        return_response = kwargs.pop("return_response", False)
        async with make_async_client(proxy=self.proxy) as client:
            response = await client.request(method, url, timeout=self.timeout, **kwargs)

        if response.status_code in {401, 403, 429}:
            raise PlatformAccessError(
                f"XHS request blocked with HTTP {response.status_code}"
            )

        if response.status_code == 471 or response.status_code == 461:
            # someday someone maybe will bypass captcha
            verify_type = response.headers["Verifytype"]
            verify_uuid = response.headers["Verifyuuid"]
            msg = f"CAPTCHA appeared, request failed, Verifytype: {verify_type}, Verifyuuid: {verify_uuid}, Response: {response}"
            utils.logger.error(msg)
            raise Exception(msg)

        response_data: Optional[Dict] = None
        try:
            candidate_data = response.json()
            if isinstance(candidate_data, dict):
                response_data = candidate_data
        except (TypeError, ValueError):
            pass

View on GitHub (pinned to d6f7c5bb90)

Solutions

  1. On 401: re-login to xhs and refresh the cookie string in config (or use --lt cookie with a fresh export).
  2. On 403: update the signing implementation (xhshow) to the latest version - the site changed its signature algorithm.
  3. On 429: reduce request rate/concurrency and enable the proxy pool (ENABLE_IP_PROXY) with a valid provider.
  4. Catch PlatformAccessError at the crawler loop level to pause/backoff instead of hammering the endpoint.

Example fix

// before
# no handling around request()
// after
from media_platform.xhs.exception import PlatformAccessError
try:
    res = await xhs_client.request("GET", uri, params=params)
except PlatformAccessError as e:
    await asyncio.sleep(60)  # backoff, then re-login/rotate proxy
    raise
Defensive patterns

Strategy: retry

Try / catch

from media_platform.xhs.exception import PlatformAccessError
try:
    res = await xhs_client.request(method, uri, **kwargs)
except PlatformAccessError as e:
    status = str(e).split()[-1]
    if status == "401":
        raise RuntimeError("xhs cookie expired - re-login required")
    await asyncio.sleep(backoff_for(status))  # 403/429: backoff + rotate proxy
    raise

Prevention

When it happens

Trigger: Any signed xhs API call (search, note detail, comments, creator notes) made with an expired web_session cookie, an x-s/x-t signature that no longer matches xiaohongshu's algorithm version, or a request burst that trips 429.

Common situations: Long crawls where the cookie expired mid-run; xiaohongshu updating their signing scheme so the local signer produces rejected x-s values; too-high concurrency or missing ENABLE_PROXY_IP causing rate limits.

Related errors

AI-assisted analysis of NanmiCoder/MediaCrawler@d6f7c5bb90 (2026-08-15). Data as JSON: /api/errors/071fea002ab48451. Report an issue: GitHub.