{"record":{"id":"33382a3cb3e12e31","repo":"unclecode/crawl4ai","slug":"failed-to-download-pdf-from-url-str-e","errorCode":null,"errorMessage":"Failed to download PDF from {url}: {str(e)}","messagePattern":"Failed to download PDF from (.+?): (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"crawl4ai/processors/pdf/__init__.py","lineNumber":188,"sourceCode":"                            progress = (downloaded / total_size) * 100\n                            if progress % 10 < 0.1:  # Log every 10%\n                                self.logger.debug(f\"PDF download progress: {progress:.0f}%\")\n                \n                if self.logger:\n                    self.logger.info(f\"PDF downloaded successfully: {temp_file.name}\")\n                        \n                return temp_file.name\n                \n            except requests.exceptions.Timeout as e:\n                # Clean up temp file if download fails\n                Path(temp_file.name).unlink(missing_ok=True)\n                self._temp_files.remove(temp_file.name)\n                raise RuntimeError(f\"Timeout downloading PDF from {url}: {str(e)}\")\n            except Exception as e:\n                # Clean up temp file if download fails\n                Path(temp_file.name).unlink(missing_ok=True)\n                self._temp_files.remove(temp_file.name)\n                raise RuntimeError(f\"Failed to download PDF from {url}: {str(e)}\")\n                \n        elif url.startswith(\"file://\"):\n            return url[7:]  # Strip file:// prefix\n            \n        return url  # Assume local path\n    \n\n__all__ = [\"PDFCrawlerStrategy\", \"PDFContentScrapingStrategy\"]","sourceCodeStart":170,"sourceCodeEnd":196,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/crawl4ai/processors/pdf/__init__.py#L170-L196","documentation":"Catch-all RuntimeError raised when any non-timeout exception occurs while the PDF crawler strategy downloads a remote PDF via requests. The temp file is cleaned up (unlink + removal from _temp_files) before raising, so disk state stays consistent. The message wraps the original exception text, which is the key to diagnosis.","triggerScenarios":"Passing an http(s) URL to the PDF download step that triggers requests exceptions other than Timeout: ConnectionError (DNS failure, refused connection), HTTPError-class failures surfaced during streaming, TLS/SSL verification errors, chunked decode errors, or disk errors writing the temp file.","commonSituations":"404/dead links, mistyped domains (DNS failure), corporate proxies or TLS interception breaking HTTPS, missing CA certificates, disk full while writing the temp file, or remote servers dropping the connection mid-download.","solutions":["Read the wrapped {str(e)} portion of the message — it identifies the root cause (DNS, TLS, HTTP status, disk).","Test the URL directly: curl -L -o /dev/null -w '%{http_code}' <url> to confirm reachability and content type.","For TLS errors, update certifi/CA bundle or disable verification only if the environment requires it.","For disk errors, free space or point TMPDIR at a volume with capacity; large PDFs can exceed small temp partitions.","If the URL requires auth or specific headers, fetch it yourself and pass a local path or file:// URL (the strategy passes those through untouched)."],"exampleFix":"# before\nresult = await crawler.arun(url=\"https://example.com/reports/report.pdf\")  # RuntimeError: Failed to download PDF ... SSLError\n\n# after\nimport requests, pathlib\nresp = requests.get(url, headers={\"Authorization\": \"Bearer ...\"}, timeout=120)\nresp.raise_for_status()\nlocal = pathlib.Path(\"/tmp/report.pdf\"); local.write_bytes(resp.content)\nresult = await crawler.arun(url=local.as_uri())  # file:// path bypasses the downloader","handlingStrategy":"try-catch","validationCode":"import requests\n\ndef check_pdf_download(url: str) -> tuple[bool, str]:\n    try:\n        r = requests.get(url, timeout=30, stream=True)\n        r.raise_for_status()\n        ct = r.headers.get(\"content-type\", \"\")\n        return (\"pdf\" in ct.lower(), ct)\n    except requests.RequestException as e:\n        return (False, str(e))","typeGuard":null,"tryCatchPattern":"try:\n    result = await crawler.arun(url=pdf_url)\nexcept RuntimeError as e:\n    if \"Failed to download PDF\" in str(e):\n        logger.error(f\"PDF fetch failed: {str(e).split(': ', 1)[-1]}\")  # root cause is in the wrapped text\n        mark_url_failed(pdf_url)","preventionTips":["Read the wrapped exception text — it distinguishes DNS, TLS, HTTP, and disk failures.","Validate URLs (scheme, domain) before enqueueing; drop obviously malformed ones.","Keep CA certificates current in containers (apt install ca-certificates / pip install -U certifi).","Ensure TMPDIR has free space for large PDFs."],"tags":["network","pdf","download","runtime-error","io"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}