{"record":{"id":"2dea6d952afd18a2","repo":"NanmiCoder/MediaCrawler","slug":"request-failed-method-method-url-url-stat","errorCode":null,"errorMessage":"Request failed, method: {method}, url: {url}, status code: {response.status_code}","messagePattern":"Request failed, method: (.+?), url: (.+?), status code: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"media_platform/tieba/client.py","lineNumber":267,"sourceCode":"        \"\"\"\n        # Check if proxy is expired before each request\n        await self._refresh_proxy_if_expired()\n\n        actual_proxy = proxy if proxy else self.default_ip_proxy\n\n        # Execute synchronous requests in thread pool\n        response = await asyncio.to_thread(\n            self._sync_request,\n            method,\n            url,\n            actual_proxy,\n            **kwargs\n        )\n\n        if response.status_code != 200:\n            utils.logger.error(f\"Request failed, method: {method}, url: {url}, status code: {response.status_code}\")\n            utils.logger.error(f\"Request failed, response: {response.text}\")\n            raise Exception(f\"Request failed, method: {method}, url: {url}, status code: {response.status_code}\")\n\n        if response.text == \"\" or response.text == \"blocked\":\n            utils.logger.error(f\"request params incorrect, response.text: {response.text}\")\n            raise Exception(\"account blocked\")\n\n        if return_ori_content:\n            return response.text\n\n        return response.json()\n\n    async def get(self, uri: str, params=None, return_ori_content=False, **kwargs) -> Any:\n        \"\"\"\n        GET request with header signing\n        Args:\n            uri: Request route\n            params: Request parameters\n            return_ori_content: Whether to return original content\n","sourceCodeStart":249,"sourceCodeEnd":285,"githubUrl":"https://github.com/NanmiCoder/MediaCrawler/blob/d6f7c5bb906b6dac40ddf343ef9e26438a3de092/media_platform/tieba/client.py#L249-L285","documentation":"Raised by BaiduTieBaClient.request (media_platform/tieba/client.py:267) when the underlying HTTP request (executed synchronously via asyncio.to_thread) returns a status code other than 200. The message includes the method, URL, and status code so the failure can be attributed to a specific endpoint. It signals the Tieba API rejecting the request — rate limiting, invalid signature/cookies, expired proxy, or server-side 5xx.","triggerScenarios":"Any client.get()/client.post() call whose HTTP response status != 200: common cases are 403/429 from anti-crawler rate limiting, 302 redirects when cookies expired, or 5xx from Tieba servers. Also triggered when a bad IP proxy makes every request fail.","commonSituations":"Crawling too fast without CRAWLER_MAX_SLEEP_SEC delays; stale or missing Tieba cookies (no login); using a dead/blocklisted proxy IP; Baidu rotating API endpoints or tightening signature checks so the signed headers no longer pass.","solutions":["Increase crawl interval (config CRAWLER_MAX_SLEEP_SEC / crawl_interval) to avoid rate limiting","Re-login via 'cookie' or 'qrcode' LOGIN_TYPE to refresh stored Tieba cookies","Rotate to a new IP proxy (configure the ip_pool) if the current IP is blocklisted","Inspect the logged response.text for the specific block/verification message and adjust headers or endpoints accordingly"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"from tenacity import retry, stop_after_attempt, wait_exponential\n\n@retry(stop=stop_after_attempt(3), wait=wait_exponential(min=2, max=30), reraise=True)\nasync def fetch_notes(client, uri, params):\n    try:\n        return await client.get(uri, params=params)\n    except Exception as e:\n        if \"status code: 4\" in str(e) or \"status code: 5\" in str(e):\n            raise  # retryable\n        raise  # non-retryable statuses surface after attempts","preventionTips":["Keep crawl_interval / CRAWLER_MAX_SLEEP_SEC high enough to avoid 403/429","Configure an ip_pool so blocked IPs can be rotated","Refresh cookies on a schedule for long-running crawls","Log response bodies (already done by the client) to classify failures"],"tags":["network","http","anti-crawl","tieba","rate-limit"],"backgroundTag":null,"analyzedSha":"d6f7c5bb906b6dac40ddf343ef9e26438a3de092","analyzedAt":"2026-08-15T01:39:07.505Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}