{"record":{"id":"d2b565beafd35e4e","repo":"D4Vinci/Scrapling","slug":"curl-arguments-parsing-error-message","errorCode":null,"errorMessage":"Curl arguments parsing error: {message}","messagePattern":"Curl arguments parsing error: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scrapling/core/shell.py","lineNumber":90,"sourceCode":"_HIDDEN_XPATH = XPath(\n    './/*[contains(@style,\"display:none\") or contains(@style,\"display: none\")'\n    ' or contains(@style,\"visibility:hidden\") or contains(@style,\"visibility: hidden\")'\n    ' or contains(@style,\"opacity:0\") or contains(@style,\"opacity: 0\")'\n    ' or contains(@style,\"font-size:0\") or contains(@style,\"font-size: 0\")'\n    ' or contains(@style,\"height:0\") or contains(@style,\"height: 0\")'\n    ' or contains(@style,\"width:0\") or contains(@style,\"width: 0\")]'\n    \" | .//*[@aria-hidden='true']\"\n    \" | .//template\"\n)\n_ZWC_PATTERN = re_compile(r\"[\\u200b\\u200c\\u200d\\ufeff\\u2060\\u180e]\")\n_CONTROL_CHARS_PATTERN = re_compile(r\"[\\x00-\\x08\\x0b\\x0c\\x0e-\\x1f]\")\n\n\n# Suppress exit on error to handle parsing errors gracefully\nclass NoExitArgumentParser(ArgumentParser):  # pragma: no cover\n    def error(self, message):\n        log.error(f\"Curl arguments parsing error: {message}\")\n        raise ValueError(f\"Curl arguments parsing error: {message}\")\n\n    def exit(self, status=0, message=None):\n        if message:\n            log.error(f\"Scrapling shell exited with status {status}: {message}\")\n            self._print_message(message, stderr)\n        raise ValueError(f\"Scrapling shell exited with status {status}: {message or 'Unknown reason'}\")\n\n\nclass CurlParser:\n    \"\"\"Builds the argument parser for relevant curl flags from DevTools.\"\"\"\n\n    def __init__(self) -> None:\n        from scrapling.fetchers import Fetcher as __Fetcher\n\n        self.__fetcher = __Fetcher\n        # We will use argparse parser to parse the curl command directly instead of regex\n        # We will focus more on flags that will show up on curl commands copied from DevTools's network tab\n        _parser = NoExitArgumentParser(add_help=False)  # Disable default help","sourceCodeStart":72,"sourceCodeEnd":108,"githubUrl":"https://github.com/D4Vinci/Scrapling/blob/5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f/scrapling/core/shell.py#L72-L108","documentation":"Scrapling's shell can ingest a curl command copied from browser DevTools by parsing it with an ArgumentParser subclass. The overridden error() method converts argparse errors (unknown flags in the defined set, missing option arguments, bad syntax) into a logged error plus ValueError instead of argparse's default sys.exit, so the failure is catchable in-process.","triggerScenarios":"Feeding a curl string to the shell/parser where a flag Scrapling maps is malformed: e.g. `-X` without a method, `--data` with no value, duplicated flags argparse can't reconcile, or unbalanced quotes that survive shlex splitting.","commonSituations":"Copy-pasting from DevTools 'Copy as cURL' then hand-editing it (dropping a value), or programmatically concatenated curl strings with quoting artifacts.","solutions":["Re-copy the exact command from DevTools (right-click request > Copy > Copy as cURL) without editing","Inspect the reported message: it names the flag and what argparse expected — restore the missing value or remove the flag","Remove unsupported/mangled flags before passing; only DevTools-common flags are mapped"],"exampleFix":"# before\nshell.from_curl(\"curl 'https://api.example.com' -X\")\n# ValueError: Curl arguments parsing error: argument -X: expected one argument\n\n# after\nshell.from_curl(\"curl 'https://api.example.com' -X POST\")","handlingStrategy":"try-catch","validationCode":"import shlex\n\ntokens = shlex.split(curl_cmd)  # raises on unbalanced quotes before scrapling sees it\nassert all(tok not in {'-h', '--help'} for tok in tokens)","typeGuard":null,"tryCatchPattern":"try:\n    req = shell.parse_curl(curl_cmd)\nexcept ValueError as e:\n    if 'Curl arguments parsing error' in str(e):\n        curl_cmd = recopy_from_devtools()  # re-copy instead of patching by hand\n        req = shell.parse_curl(curl_cmd)\n    else:\n        raise","preventionTips":["Always 'Copy as cURL' from DevTools rather than hand-writing commands","Validate shlex splitting locally before passing long edited strings"],"tags":["shell","curl","parsing","argparse"],"backgroundTag":null,"analyzedSha":"5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f","analyzedAt":"2026-08-14T22:23:09.440Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}