D4Vinci/Scrapling · error · AttributeError

Unknown/Unsupported curl arguments: {unknown}

Error message

Unknown/Unsupported curl arguments: {unknown}

What it means

After parse_known_args, any leftover tokens are flags Scrapling's curl parser does not map. Instead of silently ignoring them (which would change request semantics), it raises AttributeError('Unknown/Unsupported curl arguments: ...') listing the unknown tokens. The surrounding except block deliberately re-raises AttributeError while converting other parser failures to None.

Source

Thrown at scrapling/core/shell.py:164

        self.parser: NoExitArgumentParser = _parser
        self._supported_methods = ("get", "post", "put", "delete")

    # --- Main Parsing Logic ---
    def parse(self, curl_command: str) -> Optional[Request]:
        """Parses the curl command string into a structured context for Fetcher."""

        clean_command = curl_command.strip().lstrip("curl").strip().replace("\\\n", " ")

        try:
            tokens = shlex_split(clean_command)  # Split the string using shell-like syntax
        except ValueError as e:  # pragma: no cover
            log.error(f"Could not split command line: {e}")
            return None

        try:
            parsed_args, unknown = self.parser.parse_known_args(tokens)
            if unknown:
                raise AttributeError(f"Unknown/Unsupported curl arguments: {unknown}")

        except ValueError:  # pragma: no cover
            return None

        except AttributeError:
            raise

        except Exception as e:  # pragma: no cover
            log.error(f"An unexpected error occurred during curl arguments parsing: {e}")
            return None

        # --- Determine Method ---
        method = "get"  # Default
        if parsed_args.get:  # `-G` forces GET
            method = "get"

        elif parsed_args.method:
            method = parsed_args.method.strip().lower()

View on GitHub (pinned to 5d213a2d47)

Solutions

1. Read the listed unknown tokens and delete those flags (and their values) from the command before retrying
2. Keep the flags that matter semantically (method, headers, data, cookies, auth, proxy) and drop transport tuning flags
3. If the flag is essential (e.g. an auth header), translate it manually into the equivalent Fetcher kwarg instead of the curl string

Example fix

# before
shell.from_curl("curl 'https://example.com' --http2-only -H 'Accept: text/html'")
# AttributeError: Unknown/Unsupported curl arguments: ['--http2-only']

# after
shell.from_curl("curl 'https://example.com' -H 'Accept: text/html'")

import shlex

SUPPORTED = {'-X', '--request', '-H', '--header', '-d', '--data', '--data-raw', '-b', '--cookie', '-u', '--user', '-x', '--proxy', '-A', '--user-agent', '-e', '--referer', '-G', '--get', '--compressed', '-k', '--insecure', '-L', '--location', '-I', '--head', '--data-binary', '--data-urlencode'}

tokens = shlex.split(curl_cmd.lstrip('curl').strip())
tokens = [t for t in tokens if not t.startswith('-') or t in SUPPORTED]
curl_cmd = 'curl ' + ' '.join(tokens)

try:
    req = shell.parse_curl(curl_cmd)
except AttributeError as e:
    unknown = eval(str(e).split(': ')[-1])  # tokens listed in the message
    drop = set(unknown) | {t for t in unknown if not t.startswith('-')}
    tokens = [t for t in shlex.split(curl_cmd) if t not in drop]
    req = shell.parse_curl('curl ' + ' '.join(tokens))

When it happens

Trigger: A DevTools-copied curl containing newer or uncommon flags outside the mapped set — e.g. --http2-only, --tlsv1.3, -w '%{...}', --no-keepalive, or a typo'd flag like --cmpressed. Anything unrecognized in the token stream lands in `unknown` and triggers this.

Common situations: Chrome DevTools progressively adding flags to 'Copy as cURL' output across versions, curl commands from other tools (insomnia/postman exports) with extra options, or manually added tuning flags.

Related errors

• Curl arguments parsing error: {message}
• Scrapling shell exited with status {status}: {message or 'Un
• Input must be of type `Selector`
• Unknown extraction type: {extraction_type}
• Filename must be provided

AI-assisted analysis of D4Vinci/Scrapling@5d213a2d47 (2026-08-14). Data as JSON: /api/errors/1a7e7348200268a4. Report an issue: GitHub.