crewAIInc/crewAI · error · ImportError

`spider-client` package not found, please run `uv add spider

Error message

`spider-client` package not found, please run `uv add spider-client`

What it means

SpiderTool raises this ImportError when the spider-client package is absent and the user declines the click.confirm prompt that offers `uv pip install spider-client`. The `from None` suppresses the original import-error context; the message gives the exact manual install command.

Source

Thrown at lib/crewai-tools/src/crewai_tools/tools/spider_tool/spider_tool.py:110

        if website_url is not None:
            self.website_url = website_url

        self.log_failures = log_failures
        self.custom_params = custom_params

        try:
            from spider import Spider

        except ImportError:
            import click

            if click.confirm(
                "You are missing the 'spider-client' package. Would you like to install it?"
            ):
                subprocess.run(["uv", "pip", "install", "spider-client"], check=True)  # noqa: S607
                from spider import Spider  # type: ignore[import-untyped]
            else:
                raise ImportError(
                    "`spider-client` package not found, please run `uv add spider-client`"
                ) from None
        self.spider = Spider(api_key=api_key)

    def _validate_url(self, url: str) -> bool:
        """Validate URL format and security constraints.

        Args:
            url (str): URL to validate. Must be a properly formatted HTTP(S) URL

        Returns:
            bool: True if URL is valid and meets security requirements, False otherwise.
        """
        try:
            url = url.strip()
            decoded_url = unquote(url)

            result = urlparse(decoded_url)

View on GitHub (pinned to 754d7323be)

Solutions

Install the client manually: uv add spider-client (or pip install spider-client), then re-instantiate the tool.
Add spider-client to your project dependencies or Dockerfile so the import succeeds and the prompt never triggers.
In automation, always pre-install optional tool dependencies instead of relying on the interactive fallback.

Example fix

# terminal
# before: instantiation raises ImportError after declining prompt

# after:
#   pip install spider-client
from crewai_tools import SpiderTool
tool = SpiderTool(api_key=...)

Defensive patterns

Strategy: validation

Validate before calling

import importlib.util
if importlib.util.find_spec("spider") is None:
    raise RuntimeError("Run: pip install spider-client")

Try / catch

try:
    tool = SpiderTool(api_key=...)
except ImportError as e:
    if "spider-client" in str(e):
        subprocess.run(["pip", "install", "spider-client"], check=True)
        tool = SpiderTool(api_key=...)

Prevention

Declare spider-client in your dependencies so the interactive prompt path is never taken.
Check find_spec('spider') (the module name differs from the package name spider-client) at startup.

When it happens

Trigger: Instantiating SpiderTool without the spider package installed and answering 'no' at the interactive prompt; non-TTY environments where click.confirm cannot get input and aborts/declines.

Common situations: CI pipelines and Docker containers where no one can answer the prompt; pip-managed projects without uv; users intentionally declining auto-install to keep environments reproducible.

Related errors

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/c696c42659fd666c. Report an issue: GitHub.