{"record":{"id":"8f29f8d68fe05ff3","repo":"sherlock-project/sherlock","slug":"problem-while-attempting-to-access-data-file-url","errorCode":null,"errorMessage":"Problem while attempting to access data file URL '{data_file_path}':  {error}","messagePattern":"Problem while attempting to access data file URL '(.+?)':  (.+?)","errorType":"exception","errorClass":"FileNotFoundError","httpStatus":null,"severity":"critical","filePath":"sherlock_project/sites.py","lineNumber":129,"sourceCode":"                                  If this option is not specified, then a\n                                  default site list will be used.\n\n        Return Value:\n        Nothing.\n        \"\"\"\n\n        if not data_file_path:\n            # The default data file is the live data.json which is in the GitHub repo. The reason why we are using\n            # this instead of the local one is so that the user has the most up-to-date data. This prevents\n            # users from creating issue about false positives which has already been fixed or having outdated data\n            data_file_path = MANIFEST_URL\n\n        if data_file_path.lower().startswith(\"http\"):\n            # Reference is to a URL.\n            try:\n                response = requests.get(url=data_file_path, timeout=30)\n            except Exception as error:\n                raise FileNotFoundError(\n                    f\"Problem while attempting to access data file URL '{data_file_path}':  {error}\"\n                )\n\n            if response.status_code != 200:\n                raise FileNotFoundError(f\"Bad response while accessing \"\n                                        f\"data file URL '{data_file_path}'.\"\n                                        )\n            try:\n                site_data = response.json()\n            except Exception as error:\n                raise ValueError(\n                    f\"Problem parsing json contents at '{data_file_path}':  {error}.\"\n                )\n\n        else:\n            # Reference is to a file.\n            try:\n                with open(data_file_path, \"r\", encoding=\"utf-8\") as file:","sourceCodeStart":111,"sourceCodeEnd":147,"githubUrl":"https://github.com/sherlock-project/sherlock/blob/9100f9d40a3274bd46f4ce903c5c6fee6f3745bc/sherlock_project/sites.py#L111-L147","documentation":"SitesInformation.__init__() loads the sites manifest, defaulting to the live MANIFEST_URL (https://data.sherlockproject.xyz) when data_file_path is empty. If requests.get() raises for any reason while fetching a manifest URL, the exception is caught, wrapped and re-raised as FileNotFoundError with this message. Despite the exception type, it almost always means a network-level failure, not a missing file.","triggerScenarios":"Constructing SitesInformation() (or running the CLI) while offline; DNS failure for data.sherlockproject.xyz; a proxy/firewall blocking the egress request (requests raises ProxyError or ConnectTimeout); an SSL/TLS interception middlebox causing SSLError; a custom data_file_path that starts with 'http' but points at a dead host. The 30-second requests timeout also produces this if the server never responds.","commonSituations":"Corporate networks with mandatory proxies that reject the request; air-gapped or flaky connections; CI runners with restricted egress where data.sherlockproject.xyz is not on the allowlist; a typo'd custom --site URL (e.g. wrong hostname); the sherlock data host being temporarily down.","solutions":["Check connectivity to the manifest host first: `curl -I https://data.sherlockproject.xyz` and fix network/DNS/proxy issues (set HTTPS_PROXY if a proxy is required).","Pass a local manifest instead of the live one: `sherlock --site /path/to/data.json username` (any path not starting with http is read from disk), e.g. the copy shipped in the repo under sherlock_project/resources/.","If behind TLS interception, point REQUESTS_CA_BUNDLE at the corporate CA bundle.","If the failure is a timeout, retry — the 30s limit can trip on slow links; consider pinning a known-good manifest URL or local snapshot for reproducible runs."],"exampleFix":"# before\nsites = SitesInformation()  # fetches live manifest, fails offline\n\n# after\nimport requests\ntry:\n    sites = SitesInformation()\nexcept FileNotFoundError:\n    # fall back to the local copy shipped with the installed package\n    from importlib import resources\n    local = resources.files(\"sherlock_project\") / \"resources/data.json\"\n    sites = SitesInformation(data_file_path=str(local))","handlingStrategy":"fallback","validationCode":"import socket\nfrom urllib.parse import urlparse\n\ndef manifest_url_reachable(url: str, timeout: float = 5.0) -> bool:\n    \"\"\"Cheap DNS/TCP pre-check before constructing SitesInformation.\"\"\"\n    parsed = urlparse(url)\n    try:\n        socket.create_connection((parsed.hostname, parsed.port or 443), timeout=timeout).close()\n        return True\n    except OSError:\n        return False","typeGuard":null,"tryCatchPattern":"from sherlock_project.sites import SitesInformation\n\ntry:\n    sites = SitesInformation()  # live manifest\nexcept FileNotFoundError as err:\n    # Note: FileNotFoundError here means network failure, not a missing local file.\n    print(f\"live manifest unreachable: {err}; using local snapshot\")\n    sites = SitesInformation(data_file_path=\"sherlock_project/resources/data.json\")","preventionTips":["Cache a known-good copy of data.json locally and fall back to it when the live MANIFEST_URL is unreachable.","In CI/offline environments, always pass an explicit local --site path instead of relying on the network fetch.","Configure HTTPS_PROXY/REQUESTS_CA_BUNDLE up front in proxy or TLS-intercepting networks.","Treat FileNotFoundError from SitesInformation as a network error first — check connectivity before suspecting files."],"tags":["sherlock","network","manifest","filenotfounderror","offline","proxy"],"backgroundTag":null,"analyzedSha":"9100f9d40a3274bd46f4ce903c5c6fee6f3745bc","analyzedAt":"2026-08-14T19:48:26.535Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}