D4Vinci/Scrapling · error · ValueError

Curl arguments parsing error: {message}

Error message

Curl arguments parsing error: {message}

What it means

Scrapling's shell can ingest a curl command copied from browser DevTools by parsing it with an ArgumentParser subclass. The overridden error() method converts argparse errors (unknown flags in the defined set, missing option arguments, bad syntax) into a logged error plus ValueError instead of argparse's default sys.exit, so the failure is catchable in-process.

Source

Thrown at scrapling/core/shell.py:90

_HIDDEN_XPATH = XPath(
    './/*[contains(@style,"display:none") or contains(@style,"display: none")'
    ' or contains(@style,"visibility:hidden") or contains(@style,"visibility: hidden")'
    ' or contains(@style,"opacity:0") or contains(@style,"opacity: 0")'
    ' or contains(@style,"font-size:0") or contains(@style,"font-size: 0")'
    ' or contains(@style,"height:0") or contains(@style,"height: 0")'
    ' or contains(@style,"width:0") or contains(@style,"width: 0")]'
    " | .//*[@aria-hidden='true']"
    " | .//template"
)
_ZWC_PATTERN = re_compile(r"[\u200b\u200c\u200d\ufeff\u2060\u180e]")
_CONTROL_CHARS_PATTERN = re_compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f]")


# Suppress exit on error to handle parsing errors gracefully
class NoExitArgumentParser(ArgumentParser):  # pragma: no cover
    def error(self, message):
        log.error(f"Curl arguments parsing error: {message}")
        raise ValueError(f"Curl arguments parsing error: {message}")

    def exit(self, status=0, message=None):
        if message:
            log.error(f"Scrapling shell exited with status {status}: {message}")
            self._print_message(message, stderr)
        raise ValueError(f"Scrapling shell exited with status {status}: {message or 'Unknown reason'}")


class CurlParser:
    """Builds the argument parser for relevant curl flags from DevTools."""

    def __init__(self) -> None:
        from scrapling.fetchers import Fetcher as __Fetcher

        self.__fetcher = __Fetcher
        # We will use argparse parser to parse the curl command directly instead of regex
        # We will focus more on flags that will show up on curl commands copied from DevTools's network tab
        _parser = NoExitArgumentParser(add_help=False)  # Disable default help

View on GitHub (pinned to 5d213a2d47)

Solutions

1. Re-copy the exact command from DevTools (right-click request > Copy > Copy as cURL) without editing
2. Inspect the reported message: it names the flag and what argparse expected — restore the missing value or remove the flag
3. Remove unsupported/mangled flags before passing; only DevTools-common flags are mapped

Example fix

# before
shell.from_curl("curl 'https://api.example.com' -X")
# ValueError: Curl arguments parsing error: argument -X: expected one argument

# after
shell.from_curl("curl 'https://api.example.com' -X POST")

import shlex

tokens = shlex.split(curl_cmd)  # raises on unbalanced quotes before scrapling sees it
assert all(tok not in {'-h', '--help'} for tok in tokens)

try:
    req = shell.parse_curl(curl_cmd)
except ValueError as e:
    if 'Curl arguments parsing error' in str(e):
        curl_cmd = recopy_from_devtools()  # re-copy instead of patching by hand
        req = shell.parse_curl(curl_cmd)
    else:
        raise

When it happens

Trigger: Feeding a curl string to the shell/parser where a flag Scrapling maps is malformed: e.g. `-X` without a method, `--data` with no value, duplicated flags argparse can't reconcile, or unbalanced quotes that survive shlex splitting.

Common situations: Copy-pasting from DevTools 'Copy as cURL' then hand-editing it (dropping a value), or programmatically concatenated curl strings with quoting artifacts.

Related errors

• Scrapling shell exited with status {status}: {message or 'Un
• Unknown/Unsupported curl arguments: {unknown}
• Input must be of type `Selector`
• Unknown extraction type: {extraction_type}
• Filename must be provided

AI-assisted analysis of D4Vinci/Scrapling@5d213a2d47 (2026-08-14). Data as JSON: /api/errors/d2b565beafd35e4e. Report an issue: GitHub.