{"record":{"id":"b68e79d6492b5ca2","repo":"soxoj/maigret","slug":"problem-parsing-json-contents-at-url-str-er","errorCode":null,"errorMessage":"Problem parsing json contents at '{url}':  {str(error)}.","messagePattern":"Problem parsing json contents at '(.+?)':  (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"maigret/sites.py","lineNumber":574,"sourceCode":"        if not is_url_valid:\n            raise FileNotFoundError(f\"Invalid data file URL '{url}'.\")\n\n        import requests\n\n        try:\n            response = requests.get(url=url)\n        except Exception as error:\n            raise FileNotFoundError(\n                f\"Problem while attempting to access \"\n                f\"data file URL '{url}':  \"\n                f\"{str(error)}\"\n            )\n\n        if response.status_code == 200:\n            try:\n                data = response.json()\n            except Exception as error:\n                raise ValueError(\n                    f\"Problem parsing json contents at \" f\"'{url}':  {str(error)}.\"\n                )\n        else:\n            raise FileNotFoundError(\n                f\"Bad response while accessing \" f\"data file URL '{url}'.\"\n            )\n\n        return self.load_from_json(data)\n\n    def load_from_file(self, filename: \"str\") -> \"MaigretDatabase\":\n        try:\n            with open(filename, \"r\", encoding=\"utf-8\") as file:\n                try:\n                    data = json.load(file)\n                except Exception as error:\n                    raise ValueError(\n                        f\"Problem parsing json contents from \"\n                        f\"file '{filename}':  {str(error)}.\"","sourceCodeStart":556,"sourceCodeEnd":592,"githubUrl":"https://github.com/soxoj/maigret/blob/41674631a92a4c3ea630d046ba125cb750c03798/maigret/sites.py#L556-L592","documentation":"After a successful HTTP 200 response, load_from_http() calls response.json(); if the body is not valid JSON (requests raises json.JSONDecodeError or similar), the method raises ValueError('Problem parsing json contents at ...'). This means the URL was reachable and returned a success status, but the payload is not the expected JSON site database — e.g. an HTML error/login page, a plain-text message, or truncated content.","triggerScenarios":"Calling db.load_from_http() against a URL that returns 200 with HTML (GitHub 404-through-proxy pages, CDN interstitials, captive portal pages), a truncated download, or a JSON file with BOM/encoding corruption. Any non-JSON body with status 200 triggers it.","commonSituations":"Pointing the loader at github.com blob pages instead of raw.githubusercontent.com URLs, getting an HTML rate-limit page from a CDN, misconfigured content-type on a self-hosted copy of the data, or a partially downloaded file cached by a proxy.","solutions":["Fetch the URL manually (curl -s <url> | python -m json.tool) and inspect what the body actually is.","If it's HTML, you are probably using a rendered page URL; switch to the raw file URL (raw.githubusercontent.com/...).","If the source is genuinely corrupt, use a known-good copy (the maigret repo's resources/data.json) or load a local verified file with load_from_file().","Add a pre-check that the response Content-Type is application/json before parsing."],"exampleFix":"// before\ndb = MaigretDatabase().load_from_http('https://github.com/soxoj/maigret/blob/main/resources/data.json')  # HTML page\n\n// after\ndb = MaigretDatabase().load_from_http('https://raw.githubusercontent.com/soxoj/maigret/main/resources/data.json')  # raw JSON","handlingStrategy":"validation","validationCode":"import requests\n\nresp = requests.get(url, timeout=10)\nresp.raise_for_status()\nif 'json' not in resp.headers.get('content-type', ''):\n    raise RuntimeError(f'unexpected content-type: {resp.headers.get(\"content-type\")}')\nimport json\ndata = json.loads(resp.text)  # validate parse before maigret does\n# or: db = MaigretDatabase().load_from_json(data)","typeGuard":"import json\n\ndef is_json_body(text: str) -> bool:\n    try:\n        json.loads(text)\n        return True\n    except json.JSONDecodeError:\n        return False","tryCatchPattern":"try:\n    db = MaigretDatabase().load_from_http(url)\nexcept ValueError as err:\n    logger.error('data at %s is not valid JSON: %s', url, err)\n    db = MaigretDatabase().load_from_file(local_copy)","preventionTips":["Use raw.githubusercontent.com links, never github.com blob pages.","Check Content-Type header is application/json before parsing.","Cache a known-good JSON copy locally and fall back to it."],"tags":["json","parsing","http","maigret"],"backgroundTag":"json-parse-error","analyzedSha":"41674631a92a4c3ea630d046ba125cb750c03798","analyzedAt":"2026-08-27T02:53:58.946Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}