{"record":{"id":"88c50d2972400cc1","repo":"OpenBB-finance/OpenBB","slug":"invalid-url-url-only-urls-from-esmis-nal-usd","errorCode":null,"errorMessage":"Invalid URL '{url}'. Only URLs from 'esmis.nal.usda.gov' are supported.","messagePattern":"Invalid URL '(.+?)'\\. Only URLs from 'esmis\\.nal\\.usda\\.gov' are supported\\.","errorType":"validation","errorClass":"OpenBBError","httpStatus":null,"severity":"error","filePath":"openbb_platform/providers/government_us/openbb_government_us/models/weather_bulletin_download.py","lineNumber":65,"sourceCode":"        return GovernmentUsWeatherBulletinDownloadQueryParams(**params)\n\n    @staticmethod\n    async def aextract_data(\n        query: GovernmentUsWeatherBulletinDownloadQueryParams,\n        credentials: dict[str, Any] | None,\n        **kwargs: Any,\n    ) -> dict:\n        \"\"\"Extract the raw PDF content.\"\"\"\n        # pylint: disable=import-outside-toplevel\n        from openbb_core.provider.utils.helpers import get_async_requests_session\n\n        results: dict = {}\n        urls = query.urls\n\n        # Verify that all URLs are going to be valid USDA URLs\n        for url in urls:\n            if not url.lower().startswith(\"https://esmis.nal.usda.gov/\"):\n                raise OpenBBError(\n                    ValueError(\n                        f\"Invalid URL '{url}'. Only URLs from 'esmis.nal.usda.gov' are supported.\"\n                    )\n                )\n            if not url.lower().endswith(\".pdf\"):\n                raise OpenBBError(\n                    ValueError(\n                        f\"Invalid URL '{url}'. Only PDF documents are supported.\"\n                    )\n                )\n\n        try:\n            async with await get_async_requests_session() as session:\n                session._max_field_size = 32768  # pylint: disable=protected-access\n\n                for url in urls:\n                    async with await session.get(url) as response:\n                        if response.status != 200:","sourceCodeStart":47,"sourceCodeEnd":83,"githubUrl":"https://github.com/OpenBB-finance/OpenBB/blob/3e071fcc2cd9f891cac6040ae60296dba76dab46/openbb_platform/providers/government_us/openbb_government_us/models/weather_bulletin_download.py#L47-L83","documentation":"Guard inside weather_bulletin_download's extract_data: every URL passed via the query's 'urls' list must start with https://esmis.nal.usda.gov/. The check is a defensive SSRF-style allowlist ensuring the downloader only fetches USDA NAL ESMIS documents. Raised as OpenBBError wrapping a ValueError before any HTTP request is made.","triggerScenarios":"Passing a URL from another host (http://, another domain, or with different casing/scheme such as 'https://www.esmis.nal.usda.gov/'), or passing a URL retrieved from a different source (e.g. a search result pointing elsewhere).","commonSituations":"Copy-pasting a generic USDA link instead of the ESMIS catalog link; prefixing 'www.'; passing an empty or malformed string.","solutions":["Use canonical URLs beginning exactly with https://esmis.nal.usda.gov/ (get them from the weather_bulletin endpoint's results or the ESMIS site).","Strip 'www.' and force the https scheme before passing.","Validate URLs against the allowlist in your own code before invoking the download."],"exampleFix":"# before\nres = obb.economy.gov.weather_bulletin_download(urls=[\"https://example.com/bulletin.pdf\"])\n\n# after\nres = obb.economy.gov.weather_bulletin_download(urls=[\"https://esmis.nal.usda.gov/catalog/download/...\"])","handlingStrategy":"validation","validationCode":"ALLOWED_PREFIX = \"https://esmis.nal.usda.gov/\"\n\ndef is_usda_url(url: str) -> bool:\n    return url.lower().startswith(ALLOWED_PREFIX)","typeGuard":"def is_usda_esmis_url(u: str) -> bool is not None and isinstance(u, str) and u.lower().startswith(\"https://esmis.nal.usda.gov/\")","tryCatchPattern":null,"preventionTips":["Source URLs only from the weather_bulletin endpoint results or the ESMIS site itself.","Normalize scheme/host (strip 'www.', force https) before passing."],"tags":["validation","url-allowlist","ssrf-guard","weather-bulletin"],"backgroundTag":null,"analyzedSha":"3e071fcc2cd9f891cac6040ae60296dba76dab46","analyzedAt":"2026-08-14T23:40:48.960Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}