soxoj/maigret · error · ValueError

Problem parsing json contents at '{url}': {str(error)}.

Error message

Problem parsing json contents at '{url}':  {str(error)}.

What it means

After a successful HTTP 200 response, load_from_http() calls response.json(); if the body is not valid JSON (requests raises json.JSONDecodeError or similar), the method raises ValueError('Problem parsing json contents at ...'). This means the URL was reachable and returned a success status, but the payload is not the expected JSON site database — e.g. an HTML error/login page, a plain-text message, or truncated content.

Source

Thrown at maigret/sites.py:574

        if not is_url_valid:
            raise FileNotFoundError(f"Invalid data file URL '{url}'.")

        import requests

        try:
            response = requests.get(url=url)
        except Exception as error:
            raise FileNotFoundError(
                f"Problem while attempting to access "
                f"data file URL '{url}':  "
                f"{str(error)}"
            )

        if response.status_code == 200:
            try:
                data = response.json()
            except Exception as error:
                raise ValueError(
                    f"Problem parsing json contents at " f"'{url}':  {str(error)}."
                )
        else:
            raise FileNotFoundError(
                f"Bad response while accessing " f"data file URL '{url}'."
            )

        return self.load_from_json(data)

    def load_from_file(self, filename: "str") -> "MaigretDatabase":
        try:
            with open(filename, "r", encoding="utf-8") as file:
                try:
                    data = json.load(file)
                except Exception as error:
                    raise ValueError(
                        f"Problem parsing json contents from "
                        f"file '{filename}':  {str(error)}."

View on GitHub (pinned to 41674631a9)

Solutions

  1. Fetch the URL manually (curl -s <url> | python -m json.tool) and inspect what the body actually is.
  2. If it's HTML, you are probably using a rendered page URL; switch to the raw file URL (raw.githubusercontent.com/...).
  3. If the source is genuinely corrupt, use a known-good copy (the maigret repo's resources/data.json) or load a local verified file with load_from_file().
  4. Add a pre-check that the response Content-Type is application/json before parsing.

Example fix

// before
db = MaigretDatabase().load_from_http('https://github.com/soxoj/maigret/blob/main/resources/data.json')  # HTML page

// after
db = MaigretDatabase().load_from_http('https://raw.githubusercontent.com/soxoj/maigret/main/resources/data.json')  # raw JSON
Defensive patterns

Strategy: validation

Validate before calling

import requests

resp = requests.get(url, timeout=10)
resp.raise_for_status()
if 'json' not in resp.headers.get('content-type', ''):
    raise RuntimeError(f'unexpected content-type: {resp.headers.get("content-type")}')
import json
data = json.loads(resp.text)  # validate parse before maigret does
# or: db = MaigretDatabase().load_from_json(data)

Type guard

import json

def is_json_body(text: str) -> bool:
    try:
        json.loads(text)
        return True
    except json.JSONDecodeError:
        return False

Try / catch

try:
    db = MaigretDatabase().load_from_http(url)
except ValueError as err:
    logger.error('data at %s is not valid JSON: %s', url, err)
    db = MaigretDatabase().load_from_file(local_copy)

Prevention

When it happens

Trigger: Calling db.load_from_http() against a URL that returns 200 with HTML (GitHub 404-through-proxy pages, CDN interstitials, captive portal pages), a truncated download, or a JSON file with BOM/encoding corruption. Any non-JSON body with status 200 triggers it.

Common situations: Pointing the loader at github.com blob pages instead of raw.githubusercontent.com URLs, getting an HTML rate-limit page from a CDN, misconfigured content-type on a self-hosted copy of the data, or a partially downloaded file cached by a proxy.

Related errors

AI-assisted analysis of soxoj/maigret@41674631a9 (2026-08-27). Data as JSON: /api/errors/b68e79d6492b5ca2. Report an issue: GitHub.