{"record":{"id":"ec75044f1cbccd0c","repo":"crewAIInc/crewAI","slug":"http-response-status-response-reason","errorCode":null,"errorMessage":"HTTP {response.status}: {response.reason}","messagePattern":"HTTP (.+?): (.+?)","errorType":"http","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"lib/crewai-tools/src/crewai_tools/tools/arxiv_paper_tool/arxiv_paper_tool.py","lineNumber":89,"sourceCode":"            results = [self._format_paper_result(p) for p in papers]\n            return \"\\n\\n\" + \"-\" * 80 + \"\\n\\n\".join(results)\n\n        except Exception as e:\n            logger.error(f\"ArxivTool Error: {e!s}\")\n            return f\"Failed to fetch or download Arxiv papers: {e!s}\"\n\n    def fetch_arxiv_data(\n        self, search_query: str, max_results: int\n    ) -> list[dict[str, Any]]:\n        api_url = f\"{self.BASE_API_URL}?search_query={urllib.parse.quote(search_query)}&start=0&max_results={max_results}\"\n        logger.info(f\"Fetching data from Arxiv API: {api_url}\")\n\n        try:\n            with urllib.request.urlopen(  # noqa: S310\n                api_url, timeout=self.REQUEST_TIMEOUT\n            ) as response:\n                if response.status != 200:\n                    raise Exception(f\"HTTP {response.status}: {response.reason}\")\n                data = response.read().decode(\"utf-8\")\n        except urllib.error.URLError as e:\n            logger.error(f\"Error fetching data from Arxiv: {e}\")\n            raise\n\n        root = ET.fromstring(data)  # noqa: S314\n        papers = []\n\n        for entry in root.findall(self.ATOM_NAMESPACE + \"entry\"):\n            raw_id = self._get_element_text(entry, \"id\")\n            arxiv_id = raw_id.split(\"/\")[-1].replace(\".\", \"_\") if raw_id else \"unknown\"\n\n            title = self._get_element_text(entry, \"title\") or \"No Title\"\n            summary = self._get_element_text(entry, \"summary\") or \"No Summary\"\n            published = self._get_element_text(entry, \"published\") or \"No Publish Date\"\n            authors = [\n                self._get_element_text(author, \"name\") or \"Unknown\"\n                for author in entry.findall(self.ATOM_NAMESPACE + \"author\")","sourceCodeStart":71,"sourceCodeEnd":107,"githubUrl":"https://github.com/crewAIInc/crewAI/blob/754d7323beb2fd042e33444a115ea2d5a47193f0/lib/crewai-tools/src/crewai_tools/tools/arxiv_paper_tool/arxiv_paper_tool.py#L71-L107","documentation":"ArxivPaperTool.fetch_arxiv_data() opens the arXiv API URL with urllib and checks response.status; any non-200 status raises a generic Exception with the code and reason. In practice urllib raises HTTPError itself for most non-2xx statuses, so this branch fires mainly with non-standard status handling — it is a defensive check on the response object after urlopen succeeds.","triggerScenarios":"arXiv API returning an error status (e.g. 503 during rate limiting / maintenance windows, 400 for a malformed query string that still built a valid URL); servers or test doubles returning non-200 without urllib's default HTTPError behavior.","commonSituations":"Hammering the arXiv API without throttling and getting throttled; arXiv downtime; a search_query string with characters that survive urllib.parse.quote but still produce a bad request.","solutions":["Retry after a delay — arXiv intermittently returns 503 under load; back off (e.g. 5-15s) and reduce request frequency.","Verify the API URL by printing/logging it and opening it in a browser or with curl to see the raw status.","Respect arXiv's API etiquette: add ~3s between requests and cache results.","If persistent, check https://status.arxiv.org for outages."],"exampleFix":"# before\ntool = ArxivPaperTool()\npapers = tool.fetch_arxiv_data(\"llm agents\", 10)\n\n# after\nimport time\nfor attempt in range(3):\n    try:\n        papers = tool.fetch_arxiv_data(\"llm agents\", 10)\n        break\n    except Exception as e:\n        if attempt == 2:\n            raise\n        time.sleep(10)  # arXiv rate limits with 503s","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    papers = tool.fetch_arxiv_data(query, 10)\nexcept Exception as e:\n    if \"HTTP 503\" in str(e) or \"HTTP 429\" in str(e):\n        time.sleep(15)\n        papers = tool.fetch_arxiv_data(query, 10)\n    else:\n        raise","preventionTips":["Throttle arXiv requests (~3s spacing) and cache results — their API rate limits aggressively.","URL-encode search queries and keep them syntactically valid (field prefixes like all: or ti:).","Schedule around announced arXiv maintenance windows."],"tags":["arxiv","http","rate-limiting","external-service"],"backgroundTag":null,"analyzedSha":"754d7323beb2fd042e33444a115ea2d5a47193f0","analyzedAt":"2026-08-15T04:06:56.746Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}