{"record":{"id":"071fea002ab48451","repo":"NanmiCoder/MediaCrawler","slug":"xhs-request-blocked-with-http-response-status-cod","errorCode":null,"errorMessage":"XHS request blocked with HTTP {response.status_code}","messagePattern":"XHS request blocked with HTTP (.+?)","errorType":"exception","errorClass":"PlatformAccessError","httpStatus":null,"severity":"critical","filePath":"media_platform/xhs/client.py","lineNumber":154,"sourceCode":"        Wrapper for httpx common request method, processes request response\n        Args:\n            method: Request method\n            url: Request URL\n            **kwargs: Other request parameters, such as headers, body, etc.\n\n        Returns:\n\n        \"\"\"\n        # Check if proxy is expired before each request\n        await self._refresh_proxy_if_expired()\n\n        # return response.text\n        return_response = kwargs.pop(\"return_response\", False)\n        async with make_async_client(proxy=self.proxy) as client:\n            response = await client.request(method, url, timeout=self.timeout, **kwargs)\n\n        if response.status_code in {401, 403, 429}:\n            raise PlatformAccessError(\n                f\"XHS request blocked with HTTP {response.status_code}\"\n            )\n\n        if response.status_code == 471 or response.status_code == 461:\n            # someday someone maybe will bypass captcha\n            verify_type = response.headers[\"Verifytype\"]\n            verify_uuid = response.headers[\"Verifyuuid\"]\n            msg = f\"CAPTCHA appeared, request failed, Verifytype: {verify_type}, Verifyuuid: {verify_uuid}, Response: {response}\"\n            utils.logger.error(msg)\n            raise Exception(msg)\n\n        response_data: Optional[Dict] = None\n        try:\n            candidate_data = response.json()\n            if isinstance(candidate_data, dict):\n                response_data = candidate_data\n        except (TypeError, ValueError):\n            pass","sourceCodeStart":136,"sourceCodeEnd":172,"githubUrl":"https://github.com/NanmiCoder/MediaCrawler/blob/d6f7c5bb906b6dac40ddf343ef9e26438a3de092/media_platform/xhs/client.py#L136-L172","documentation":"Raised by XiaoHongShuClient.request when xiaohongshu answers with HTTP 401, 403, or 429. These statuses mean the platform actively blocked the request: 401 invalid/expired session cookie, 403 forbidden (signature or cookie rejected), 429 rate limited. It is a PlatformAccessError, distinct from data errors, so callers can branch on access-level failures.","triggerScenarios":"Any signed xhs API call (search, note detail, comments, creator notes) made with an expired web_session cookie, an x-s/x-t signature that no longer matches xiaohongshu's algorithm version, or a request burst that trips 429.","commonSituations":"Long crawls where the cookie expired mid-run; xiaohongshu updating their signing scheme so the local signer produces rejected x-s values; too-high concurrency or missing ENABLE_PROXY_IP causing rate limits.","solutions":["On 401: re-login to xhs and refresh the cookie string in config (or use --lt cookie with a fresh export).","On 403: update the signing implementation (xhshow) to the latest version - the site changed its signature algorithm.","On 429: reduce request rate/concurrency and enable the proxy pool (ENABLE_IP_PROXY) with a valid provider.","Catch PlatformAccessError at the crawler loop level to pause/backoff instead of hammering the endpoint."],"exampleFix":"// before\n# no handling around request()\n// after\nfrom media_platform.xhs.exception import PlatformAccessError\ntry:\n    res = await xhs_client.request(\"GET\", uri, params=params)\nexcept PlatformAccessError as e:\n    await asyncio.sleep(60)  # backoff, then re-login/rotate proxy\n    raise","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"from media_platform.xhs.exception import PlatformAccessError\ntry:\n    res = await xhs_client.request(method, uri, **kwargs)\nexcept PlatformAccessError as e:\n    status = str(e).split()[-1]\n    if status == \"401\":\n        raise RuntimeError(\"xhs cookie expired - re-login required\")\n    await asyncio.sleep(backoff_for(status))  # 403/429: backoff + rotate proxy\n    raise","preventionTips":["Monitor cookie age and refresh before expiry on long runs.","Rate-limit xhs requests below the 429 threshold.","Keep the xhshow signer updated when xiaohongshu rotates its signature scheme.","Branch on the status embedded in the message: 401=cookie, 403=signature/blocked, 429=rate."],"tags":["xhs","anti-bot","rate-limit","signing","http"],"backgroundTag":null,"analyzedSha":"d6f7c5bb906b6dac40ddf343ef9e26438a3de092","analyzedAt":"2026-08-15T01:39:07.505Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}