scrapy/scrapy · error · UsageError

Invalid --cbkwargs value, pass a valid json string to --cbkw

Error message

Invalid --cbkwargs value, pass a valid json string to --cbkwargs. Example: --cbkwargs='{"foo" : "bar"}'

What it means

'scrapy parse' accepts --cbkwargs as a JSON object passed as cb_kwargs to the request callback. json.loads() failures (ValueError) are converted to this UsageError. Like -m/--meta, the input must be strict JSON.

Source

Thrown at scrapy/commands/parse.py:397

        self.process_request_cb_kwargs(opts)

    def process_request_meta(self, opts: argparse.Namespace) -> None:
        if opts.meta:
            try:
                opts.meta = json.loads(opts.meta)
            except ValueError:
                raise UsageError(
                    "Invalid -m/--meta value, pass a valid json string to -m or --meta. "
                    'Example: --meta=\'{"foo" : "bar"}\'',
                    print_help=False,
                ) from None

    def process_request_cb_kwargs(self, opts: argparse.Namespace) -> None:
        if opts.cbkwargs:
            try:
                opts.cbkwargs = json.loads(opts.cbkwargs)
            except ValueError:
                raise UsageError(
                    "Invalid --cbkwargs value, pass a valid json string to --cbkwargs. "
                    'Example: --cbkwargs=\'{"foo" : "bar"}\'',
                    print_help=False,
                ) from None

    def run(self, args: list[str], opts: argparse.Namespace) -> None:
        # parse arguments
        if not len(args) == 1 or not is_url(args[0]):
            raise UsageError
        url = args[0]

        # prepare spidercls
        self.set_spidercls(url, opts)

        if self.spidercls and opts.depth > 0:
            self.start_parsing(url, opts)
            self.print_results(opts)

View on GitHub (pinned to 06af687662)

Solutions

1. Quote the JSON object and use double-quoted keys: scrapy parse <url> --cbkwargs='{"page": 2}'.
2. Pre-validate the payload with json.loads() in the tool/language building the command.
3. Remember only JSON scalar/object/array types are allowed; convert Python booleans to true/false.

Example fix

# before
scrapy parse https://example.com --cbkwargs "{page: 2}"

# after
scrapy parse https://example.com --cbkwargs='{"page": 2}'

import json

kwargs_str = '{"page": 2}'
cb_kwargs = json.loads(kwargs_str)
assert isinstance(cb_kwargs, dict)

When it happens

Trigger: Passing '--cbkwargs {page: 2}' (unquoted key), Python-style True/None, trailing commas, or a shell-mangled argument where quotes were stripped.

Common situations: Testing callbacks that require keyword arguments; shell quoting issues (esp. zsh/cmd.exe) when embedding JSON with double quotes in the argument.

Related errors

• Invalid -m/--meta value, pass a valid json string to -m or -
• Cannot find callback {cb!r} in spider: {spider.name}
• Invalid -s value, use -s NAME=VALUE
• Invalid -a value, use -a NAME=VALUE
• running 'scrapy crawl' with more than one spider is not supp

AI-assisted analysis of scrapy/scrapy@06af687662 (2026-08-15). Data as JSON: /api/errors/9b161ee63cec0110. Report an issue: GitHub.