unclecode/crawl4ai · error · RuntimeError

Timeout downloading PDF from {url}: {str(e)}

Error message

Timeout downloading PDF from {url}: {str(e)}

What it means

Raised when a requests.exceptions.Timeout escapes while the PDF crawler strategy streams a remote PDF to a temp file. It is re-raised as RuntimeError after the temp file is unlinked and removed from the internal _temp_files list, so no partial download leaks. The message embeds the offending URL and the underlying timeout exception text.

Source

Thrown at crawl4ai/processors/pdf/__init__.py:183

                with open(temp_file.name, 'wb') as f:
                    for chunk in response.iter_content(chunk_size=8192):
                        f.write(chunk)
                        downloaded += len(chunk)
                        if self.logger and total_size > 0:
                            progress = (downloaded / total_size) * 100
                            if progress % 10 < 0.1:  # Log every 10%
                                self.logger.debug(f"PDF download progress: {progress:.0f}%")
                
                if self.logger:
                    self.logger.info(f"PDF downloaded successfully: {temp_file.name}")
                        
                return temp_file.name
                
            except requests.exceptions.Timeout as e:
                # Clean up temp file if download fails
                Path(temp_file.name).unlink(missing_ok=True)
                self._temp_files.remove(temp_file.name)
                raise RuntimeError(f"Timeout downloading PDF from {url}: {str(e)}")
            except Exception as e:
                # Clean up temp file if download fails
                Path(temp_file.name).unlink(missing_ok=True)
                self._temp_files.remove(temp_file.name)
                raise RuntimeError(f"Failed to download PDF from {url}: {str(e)}")
                
        elif url.startswith("file://"):
            return url[7:]  # Strip file:// prefix
            
        return url  # Assume local path
    

__all__ = ["PDFCrawlerStrategy", "PDFContentScrapingStrategy"]

View on GitHub (pinned to 7e80152142)

Solutions

  1. Increase the download timeout in the PDF crawler strategy / crawl config (e.g. set a larger timeout passed to requests for PDF downloads).
  2. Retry with backoff — transient network stalls often succeed on a second attempt.
  3. Verify the URL is reachable (curl -I) and that it actually serves a PDF, not a login page or redirect chain that hangs.
  4. If the host is slow by design, fetch the PDF yourself with a longer/no timeout and pass a local file:// path or local file path instead of the remote URL.

Example fix

# before
result = await crawler.arun(url="https://slow-host.example/report.pdf")  # RuntimeError: Timeout downloading PDF

# after
# raise the timeout budget for the PDF fetch (strategy kwargs / config)
pdf_strategy = PDFCrawlerStrategy(timeout=120)  # or set in config: pdf_download_timeout = 120
result = await crawler.arun(url="https://slow-host.example/report.pdf")
Defensive patterns

Strategy: retry

Validate before calling

import requests

def pdf_url_reachable(url: str, timeout: float = 10) -> bool:
    try:
        with requests.head(url, timeout=timeout, allow_redirects=True) as r:
            return r.status_code == 200 and "pdf" in r.headers.get("content-type", "").lower()
    except requests.RequestException:
        return False

Try / catch

try:
    result = await crawler.arun(url=pdf_url)
except RuntimeError as e:
    if "Timeout downloading PDF" in str(e):
        # backoff and retry, or raise the configured timeout before next attempt
        raise

Prevention

When it happens

Trigger: Calling the PDF crawler strategy's download step with an http(s) URL whose server does not respond within the requests timeout (connect or read timeout) configured for the download. Only the requests.exceptions.Timeout branch produces this message; generic failures raise the sibling 'Failed to download PDF' error.

Common situations: Crawling PDFs from slow or overloaded hosts, proxies or CDNs that stall mid-stream, low timeout settings in the PDF download config, or networks blocking large file transfers. Also happens when the URL points to a host that accepts connections but never finishes sending the body.

Understand the failure class

Related errors

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/c8f50fd90a6c0ae0. Report an issue: GitHub.