OpenBB-finance/OpenBB · error · OpenBBError

Failed to download document from

Error message

Failed to download document from {url}. Status code: {response.status}

What it means

Raised during the aiohttp download loop when response.status is not 200 for an allowlisted PDF URL. The generic except Exception at the end of extract_data re-wraps it into OpenBBError, so the caller sees this message regardless of the underlying status. It means the URL passed validation but the server did not return the file.

Solutions

Verify the URL opens in a browser; if 404, re-fetch a fresh URL from the weather_bulletin endpoint.
For 429/5xx, reduce batch size and retry after a delay.
Check that only existing bulletin dates/years are requested.

Defensive patterns

Strategy: retry

Try / catch

try:
    res = obb.economy.gov.weather_bulletin_download(urls=[u])
except OpenBBError as e:
    if "Failed to download document" in str(e):
        time.sleep(5)
        res = obb.economy.gov.weather_bulletin_download(urls=[u])
    else:
        raise

Prevention

Re-fetch fresh URLs from weather_bulletin instead of storing them long-term.
Batch downloads modestly and add delays to avoid USDA throttling.
Verify each URL with a HEAD request before the bulk call.

When it happens

Trigger: A valid-looking esmis.nal.usda.gov/*.pdf URL that has been moved or removed (404), server-side errors (5xx), or rate limiting (429) when downloading several bulletins in one call.

Common situations: Stale URLs harvested long ago; batch downloads of many bulletins triggering throttling; ESMIS maintenance windows.

Related errors

AI-assisted analysis of OpenBB-finance/OpenBB@3e071fcc2c (2026-08-14). Data as JSON: /api/errors/08511aee507f286e. Report an issue: GitHub.

Appendix: source

Thrown at openbb_platform/providers/government_us/openbb_government_us/models/weather_bulletin_download.py:84

                    ValueError(
                        f"Invalid URL '{url}'. Only URLs from 'esmis.nal.usda.gov' are supported."
                    )
                )
            if not url.lower().endswith(".pdf"):
                raise OpenBBError(
                    ValueError(
                        f"Invalid URL '{url}'. Only PDF documents are supported."
                    )
                )

        try:
            async with await get_async_requests_session() as session:
                session._max_field_size = 32768  # pylint: disable=protected-access

                for url in urls:
                    async with await session.get(url) as response:
                        if response.status != 200:
                            raise OpenBBError(
                                ValueError(
                                    f"Failed to download document from {url}. Status code: {response.status}"
                                )
                            )
                        content_bytes = await response.read()
                        results[url] = content_bytes

            return results
        except Exception as e:
            raise OpenBBError(e) from e

    @staticmethod
    def transform_data(
        query: GovernmentUsWeatherBulletinDownloadQueryParams,
        data: dict,
        **kwargs: Any,
    ) -> list[GovernmentUsWeatherBulletinDownloadData]:
        """Transform the raw PDF content into the data model."""

View on GitHub (pinned to 3e071fcc2c)