D4Vinci/Scrapling · error · ValueError

Scrapling shell exited with status {status}: {message or 'Un

Error message

Scrapling shell exited with status {status}: {message or 'Unknown reason'}

What it means

The same NoExitArgumentParser also overrides exit(); argparse calls exit() after help/version actions or after error(). The override logs the status/message and re-raises it as ValueError ('Unknown reason' when message is empty) so the shell never hard-exits the process. Users typically hit it when the pasted curl command contains -h/--help or triggers another exit path.

Source

Thrown at scrapling/core/shell.py:96

    ' or contains(@style,"width:0") or contains(@style,"width: 0")]'
    " | .//*[@aria-hidden='true']"
    " | .//template"
)
_ZWC_PATTERN = re_compile(r"[\u200b\u200c\u200d\ufeff\u2060\u180e]")
_CONTROL_CHARS_PATTERN = re_compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f]")


# Suppress exit on error to handle parsing errors gracefully
class NoExitArgumentParser(ArgumentParser):  # pragma: no cover
    def error(self, message):
        log.error(f"Curl arguments parsing error: {message}")
        raise ValueError(f"Curl arguments parsing error: {message}")

    def exit(self, status=0, message=None):
        if message:
            log.error(f"Scrapling shell exited with status {status}: {message}")
            self._print_message(message, stderr)
        raise ValueError(f"Scrapling shell exited with status {status}: {message or 'Unknown reason'}")


class CurlParser:
    """Builds the argument parser for relevant curl flags from DevTools."""

    def __init__(self) -> None:
        from scrapling.fetchers import Fetcher as __Fetcher

        self.__fetcher = __Fetcher
        # We will use argparse parser to parse the curl command directly instead of regex
        # We will focus more on flags that will show up on curl commands copied from DevTools's network tab
        _parser = NoExitArgumentParser(add_help=False)  # Disable default help
        # Basic curl arguments
        _parser.add_argument("curl_command_placeholder", nargs="?", help=SUPPRESS)
        _parser.add_argument("url")
        _parser.add_argument("-X", "--request", dest="method", default=None)
        _parser.add_argument("-H", "--header", action="append", default=[])
        _parser.add_argument(

View on GitHub (pinned to 5d213a2d47)

Solutions

1. Strip help flags from the curl string before parsing
2. Check the logged status/message pair — status 0 with usage text means a help action fired, not a real failure
3. Re-copy the original DevTools curl command

Example fix

# before
shell.from_curl("curl 'https://example.com' -h")
# ValueError: Scrapling shell exited with status 0: ...

# after
shell.from_curl("curl 'https://example.com'")

curl_cmd = ' '.join(t for t in curl_cmd.split() if t not in {'-h', '--help'})

try:
    req = shell.parse_curl(curl_cmd)
except ValueError as e:
    if 'exited with status' in str(e):
        raise ValueError('curl command contained a help/exit flag') from e
    raise

When it happens

Trigger: A parsed curl command containing help-like flags (-h, --help) that make argparse print usage and call exit(0); or any argparse code path that finishes via exit() with a message.

Common situations: Pasting an edited curl that gained a stray `-h`, or passing a raw command that begins with something argparse treats as an action requesting exit.

Related errors

• Curl arguments parsing error: {message}
• Unknown/Unsupported curl arguments: {unknown}
• Input must be of type `Selector`
• Unknown extraction type: {extraction_type}
• Filename must be provided

AI-assisted analysis of D4Vinci/Scrapling@5d213a2d47 (2026-08-14). Data as JSON: /api/errors/ee6b1bc68a1eef2b. Report an issue: GitHub.