D4Vinci/Scrapling · error · ValueError

Invalid JSON data '{json_string}': {err}

Error message

Invalid JSON data '{json_string}': {err}

What it means

The CLI's --json option expects a raw JSON string (an object of body parameters). __ParseJSONData runs orjson.loads on it and raises ValueError with the decoder message when the string is not valid JSON. This is strict input validation before the request is made; nothing is sent on the wire when it fires.

Source

Thrown at scrapling/cli.py:39

__OUTPUT_FILE_HELP__ = "The output file path can be an HTML file, a Markdown file of the HTML content, or the text content itself. Use file extensions (`.html`/`.md`/`.txt`) respectively."
__PACKAGE_DIR__ = Path(__file__).parent


def __Execute(cmd: List[str], help_line: str) -> None:  # pragma: no cover
    print(f"Installing {help_line}...")
    _ = check_output(cmd, shell=False)  # nosec B603
    # I meant to not use try except here


def __ParseJSONData(json_string: Optional[str] = None) -> Optional[Dict[str, Any]]:
    """Parse JSON string into a Python object"""
    if not json_string:
        return None

    try:
        return json_loads(json_string)
    except JSONDecodeError as err:  # pragma: no cover
        raise ValueError(f"Invalid JSON data '{json_string}': {err}")


def __Request_and_Save(
    fetcher_func: Callable[..., Response],
    url: str,
    output_file: str,
    css_selector: Optional[str] = None,
    ai_targeted: bool = False,
    **kwargs,
) -> None:
    """Make a request using the specified fetcher function and save the result"""
    from scrapling.core.shell import Convertor

    # Handle relative paths - convert to an absolute path based on the current working directory
    output_path = Path(output_file)
    if not output_path.is_absolute():
        output_path = Path.cwd() / output_file

View on GitHub (pinned to 5d213a2d47)

Solutions

1. Validate the payload locally first: `echo '<payload>' | python -m json.tool` (or `jq .`) to get the exact parse error
2. Quote JSON keys and the whole string: `--json '{"key": "value"}'`
3. Use single quotes around the JSON in the shell so $ and spaces survive; for data with single quotes, switch to a file or use jq to build it
4. If the data is key=value pairs, use the params option instead of --json

Example fix

# before
scrapling fetch https://api.example.com --json "username: test"
# ValueError: Invalid JSON data 'username: test': ...

# after
scrapling fetch https://api.example.com --json '{"username": "test"}'

import json

json_str = '{"key": "value"}'
json.loads(json_str)  # raises locally with a precise message before the CLI does

try:
    run_cli_fetch(url, json=payload)
except ValueError as e:
    if 'Invalid JSON data' in str(e):
        payload = json.dumps(json.loads(payload))  # rebuild cleanly and retry once
    else:
        raise

When it happens

Trigger: Passing a non-JSON string to `scrapling fetch URL --json ...`, e.g. `--json 'a=b'`, `--json "{key: 1}"` (unquoted keys), or truncated JSON. Shell quoting mistakes (single vs double quotes around $variables) that corrupt the payload also land here.

Common situations: Building CLI commands in shell scripts where interpolation breaks quotes, hand-typing JSON without quoting keys, or passing form-encoded data (`k=v`) to the JSON flag instead of the params flag.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Could not parse cookies '{cookies}': {err}
• You need to install scrapling with any of the extras to enab
• Credentials dictionary must contain both 'username' and 'pas
• Session '{session_id}' already exists. Use a different ID or
• Filename must be provided

AI-assisted analysis of D4Vinci/Scrapling@5d213a2d47 (2026-08-14). Data as JSON: /api/errors/843446e105ab4e8b. Report an issue: GitHub.