assafelovic/gpt-researcher · error · ValueError

🤷 Failed to load any documents!

Error message

🤷 Failed to load any documents!

What it means

OnlineDocumentLoader.load fetches documents from URLs; if no URL produced any parsed content (all downloads/parses failed or the list was empty), docs is empty and this ValueError is raised.

Source

Thrown at gpt_researcher/document/online_document.py:34

class OnlineDocumentLoader:

    def __init__(self, urls):
        self.urls = urls

    async def load(self) -> list:
        docs = []
        for url in self.urls:
            pages = await self._download_and_process(url)
            for page in pages:
                if page.page_content:
                    docs.append({
                        "raw_content": page.page_content,
                        "url": page.metadata.get("source")
                    })

        if not docs:
            raise ValueError("🤷 Failed to load any documents!")

        return docs

    async def _download_and_process(self, url: str) -> list:
        try:
            # Reject SSRF / local-file targets before issuing the request.
            try:
                validate_url(url)
            except UnsafeURLError as e:
                print(f"Skipping unsafe document URL {url}: {e}")
                return []

            headers = {
                "User-Agent": "Mozilla/5.0"
            }
            async with aiohttp.ClientSession() as session:
                async with session.get(url, headers=headers, timeout=6) as response:
                    if response.status != 200:

View on GitHub (pinned to 6f998577d5)

Solutions

1. Check each URL returns 200 and real content (curl/browser) before loading
2. Filter out non-document links; verify at least one source is a parseable doc
3. Handle per-URL failures upstream and require >=1 valid source before calling load

Example fix

# before
loader = OnlineDocumentLoader(['https://example.com/missing.pdf'])
# after
valid = [u for u in urls if requests.head(u, timeout=10).ok]
if not valid: raise SystemExit('no valid sources')
loader = OnlineDocumentLoader(valid)

import requests
ok = [u for u in urls if u.startswith('http') and requests.head(u, timeout=10, allow_redirects=True).status_code < 400]
assert ok, 'no reachable document URLs'

try:
    docs = loader.load()
except ValueError as e:
    if "Failed to load any documents" in str(e):
        docs = fallback_scrape(urls)
    else: raise

When it happens

Trigger: Passing URLs that 404/timeout, HTML pages with no extractable text, non-document content-types, or an empty list of sources.

Common situations: Scraped links behind auth/JS rendering; typo'd URLs; sources returning HTML error pages that parse to nothing.

Related errors

• 🤷 Failed to load any documents!
• Invalid type for path. Expected str, bytes, os.PathLike, or
• Cost must be an integer or float
• Embedding provider not found.
• Invalid retriever(s) found: {', '.join(invalid_retrievers)}.

AI-assisted analysis of assafelovic/gpt-researcher@6f998577d5 (2026-08-28). Data as JSON: /api/errors/352fb12b7d5b8959. Report an issue: GitHub.